Deep Agents v0

Masonry Sixty-five percent fewer tokens and performance held. That's not a footnote, that's the whole pitch.

Eyre It is, but I want to be precise about what 'held' means here, because they're actually pretty honest about it. The reward confidence intervals span zero for every model. So directionally the story is good, Luna is the clearest case — thirty-four percent token reduction, fifteen percent cost drop, reward up four percent — but they're not overclaiming statistical certainty across the board.

Masonry Right.

Eyre And I actually respect that they said it. A lot of harness releases would just lead with the Luna number and bury the caveat. They put it in the body.

Masonry Okay so walk me through what they actually cut, because three changes for sixty-five percent feels like there was a lot of fat in there.

Eyre There was. They removed the base system prompt entirely — general guidelines, tool-usage prose, gone. Trimmed the builtin tool descriptions by forty-three percent. And TodoListMiddleware is no longer on by default. Put those together and you go from roughly six thousand base input tokens per turn down to about two thousand.

Masonry Oh interesting.

Eyre The Anthropic context engineering paper is the direct antecedent here. LangChain even cites it — Anthropic cut over eighty percent of the Claude Code system prompt for Opus 5 and Fable 5 with no measurable drop in coding evals. The two findings that Deep Agents says mirror their own work: good tool schemas teach usage better than few-shot examples, and repeating an instruction in both the system prompt AND a tool description doesn't reinforce anything, it just adds tokens.

Masonry We literally talked about this exact pattern last episode. The harness gets simpler as the model gets stronger.

Eyre Yeah, it keeps showing up. And the v0.7 eval suite is actually interesting on its own — three categories: autonomous tasks like coding and data analysis, conversational multi-turn, and long-context retrieval and reasoning. Four models: GPT-5.6 Luna, Gemini 3.6 Flash, Claude Sonnet 4.6, Claude Opus 4.8. That's a real matrix, not a cherry-picked single-model benchmark.

Masonry Sure.

Eyre But Masonry, the Sonnet result is the one I keep coming back to. Cost went UP on Sonnet 4.6. Not down.

Masonry Yeah, that jumped out at me too. What happened there?

Eyre They dug into the LangSmith traces and traced it to two specific challenging autonomous tasks. Not a general regression — two hard problems where the leaner harness apparently let the model run longer before getting unstuck, which cost more. The diagnosis only exists because the traces existed.

Masonry Okay, that is a perfect little case study for the oversight infrastructure point we keep making. The number looked bad. The trace told you WHY. Without LangSmith in the loop that finding just reads as 'Sonnet got worse,' and you ship a rollback.

Eyre Exactly. You'd never know if it was the harness change or something about those specific task types. The visibility is what makes the result actionable instead of just confusing.

Masonry Okay, I want to talk about the configurability piece because that might actually be the bigger unlock for teams already using this. What did they ship there?

Eyre So the top user request over the last six months was basically 'let me override the default middleware without doing something hacky.' There's even a quote in the post from a power user: they were doing, quote, 'some hacky stuff to remove some of the default middleware.' That's not a power user problem, that's a harness design problem.

Masonry No way, it was just erroring before?

Eyre And the reason removing the hidden system prompt HELPS with this is subtle but real. If there's prompting under the hood you can't see, your custom prompting is fighting an invisible opponent. Conflicts, redundancies — you don't know what's winning. Clean slate means your prompts are actually in charge.

Masonry Mm-hm.

Masonry Alright, the todo list thing. They made it opt-in. I have mixed feelings about this one, actually.

Eyre The evals are pretty clear — slightly better rewards AND lower cost with todos disabled, across three categories and three models. The planning prompt and write_todos tool just weren't earning their token budget on average.

Masonry Right, but the carve-outs they list are the honest part of the story. Long multi-step tasks, less capable models, UI-facing cases where a visible plan matters. Those aren't edge cases for a lot of teams.

Eyre No, they're not. And this is where I think the 'harness gets simpler' pattern has a real boundary condition. It holds when you're running a capable model on a well-scoped task. Stack a weaker model onto a long multi-step workflow and suddenly the scaffolding is load-bearing again.

Masonry Yeah. We said something similar about this in the context of routing — not every task wants the same setup. The effort dial we talked about with Opus 5 applies here too. The harness config IS part of the deployment decision.

Eyre And the one-liner to turn todos back on is just middleware equals TodoListMiddleware in brackets. So the friction of opting back in is basically zero.

Masonry Okay I do want to flag one thing before we move on — how are you feeling about the eval suite itself? Three categories, four models, that sounds thorough, but…

Eyre The confidence interval thing is my main hedge. Luna and Opus show statistically clear token reductions. Luna also shows a statistically clear cost reduction. But for reward, the intervals span zero for every model. Which means they can't actually rule out that reward stayed flat by luck on some of these. It's honest reporting, I just want people reading the headline number to know that 'comparable performance' is a directional claim, not a proven equivalence.

Masonry That is such an Eyre read of a release post.

Eyre I mean… they PUT it in the post. I'm just reading it.

Masonry Okay, that's fair. That's fair. And honestly the filesystem changes are kind of quietly useful too — write_file now overwrites instead of erroring, paginated reads report remaining lines and the next offset, grep gets a thousand-match cap and streamed output. None of that is glamorous but it's the kind of thing that makes agents not fall over on real codebases.

Eyre The grep truncation flag especially. If you've ever had an agent hang on a large repo because glob returned ten thousand paths and nobody put a ceiling on it… yeah. That fix is worth a release on its own.

Masonry Okay, if you're on this already — it's deepagents on PyPI, uv pip install deepagents, or npm install deepagents at latest for the JS side. The changelog has migration notes and they apparently include an upgrade prompt for coding agents, which is a nice touch.

Eyre The eval suite blog is also worth reading separately — Nick Hollon and Harrison Chase published it a week earlier. It's how they benchmark, and it's more methodologically interesting than most harness release posts bother to be.

Masonry Alright Eyre, I think this is genuinely one of the cleaner releases we've covered. Not everything ships with the confidence intervals included.