Introducing Grok 4

Pippa I think the disagreement is whether Grok 4.6 is a real step for people trying to get work done, or just another frontier score wearing a tiny product hat.

Tyler My week's been weirdly full of launch posts, so this one landed at exactly the moment when every model claims it can run a whole project. I'm skeptical, mostly because long-running agents are where demos go to become invoices.

Tyler The headline is that Grok 4.6 matches GPT-5.6 Sol on the Artificial Analysis Intelligence Index at sixty-one. Fine. That's a composite of nine benchmarks, and the detailed table is much less tidy. Grok beats Sol on CursorBench, but trails it on DeepSWE, Terminal-Bench, and APEX-Agents.

Pippa Right.

Tyler So I don't read this as Grok 4.6 being generally equal to Sol. I read it as a model with a particular shape of competence, plus a launch page that wisely puts the composite number above the awkward parts.

Pippa That pushback is fair, Tyler, but the shape is the interesting bit. The release is aimed at staying with a task across research, a codebase, or a visual application. That's a real pain. Most tools can make a decent first move, then lose the thread when the work becomes a sequence of decisions.

Tyler Mm-hm.

Pippa The user story is somebody with a broad product idea who wants a substantial first version instead of a blank canvas and six disconnected code snippets. Grok 4.6 is supposed to establish the structure and visual language, implement interactions, then keep refining after feedback. That's much closer to a workflow than a chatbot feature.

Tyler I buy the workflow framing more than the benchmark framing. The technical story is also more than a token bump. They ran a longer supplemental training phase, used curated model-generated data, improved the optimizer and recipe, then had Grok 4.5 regenerate supervised trajectories across reasoning effort, agent harnesses, software engineering, and knowledge work.

Pippa Oh interesting.

Tyler They filtered problematic traces with model-based checks and followed that with reinforcement learning across coding, knowledge work, kernel optimization, web development, and computer-aided design. That could explain better persistence and self-testing. But it also means the evidence depends heavily on how those environments reward behavior. A model can learn to look diligent inside the harness without becoming dependable outside it.

Pippa You say that like the model is secretly submitting expense reports to its own evaluator.

Tyler I mean, if the evaluator has a checkbox for verification, the model will discover the checkbox. That's the whole reason I care about whether the self-testing is actually catching errors or just generating a ceremonial test pass.

Pippa Sure.

Tyler And the numbers support caution. Grok 4.6 gets sixty-five point nine percent on DeepSWE, versus seventy-three percent for Sol. On Terminal-Bench it gets twenty-six percent, while Sol is at thirty-four point six. Those are exactly the environments where sustained tool use and recovery matter, not just polished answers.

Pippa Okay, tiny tangent. The phrase visual language in a model release always makes me imagine an agent choosing a font and then standing back like it just painted a mural.

Pippa And honestly, I want that agent. Not because typography is profound, but because starting with a coherent interface is useful. The fastest route to a good product can be a strong rough version that gives a human something concrete to criticize.

Tyler Exactly. That's where I think the release earns some attention. It isn't promising an autonomous company in a box. It is betting that a model can reduce the cost of getting from vague intent to an editable artifact.

Pippa And the adoption path is unusually clear. It's already in Cursor and Grok Build, with twice the included usage for the first week. Then it is in the API and partner platforms including OpenRouter, Vercel, and Cloudflare. Developers don't have to invent a new interface just to test the claim.

Tyler That matters, although the incentive can blur retention. The API starts at two dollars per million input tokens and six dollars per million output tokens, with a fast variant at twice the price. That's cheap enough for experimentation, but long-running agents can burn through cheap tokens very efficiently.

Pippa Right, right.

Tyler My honest read is positive but narrow. Grok 4.6 looks like a serious attempt to improve the model-harness loop, especially for interactive builds. I don't think the public evidence supports calling it the new default for coding or agent work, because the benchmark profile is uneven and the launch examples are still curated.

Pippa I'm a little more bullish. If the model reliably makes a useful first pass and gets better through a few feedback cycles, that can win even without owning every benchmark. The thing I'd watch is whether people keep it in Cursor or Grok Build after the two-times usage week ends.

Tyler For a hands-on test, the launch page gives you the Grok CLI install command, the API key path, and the model docs. The command is curl dash f s s L, pointing to x dot ai slash C L I slash install dot sh, then piping into bash. I would run that in a disposable environment, give it a small application brief, and inspect every file and external action.

Pippa That is the most Tyler possible build recommendation. Disposable environment, inspect everything, trust nothing.

Tyler It has kept us from recording several very confident episodes about broken demos.

Pippa Anyway, I cannot believe episode eight hundred fifty-six is ending with us debating whether an agent picked a good font. That's a real product test now, apparently. I'll see you in the next one, Tyler.