Harness Zero: Harness Distillation via Agent as Harness

Onyx So this paper basically says, what if your harness optimizations didn’t have to stay glued to the harness forever.

Echo Yeah, it’s like, you do all this fancy harness engineering and evolution… and Harness-Zero asks, why aren’t we turning that into model weights.

Onyx Right. Agent equals model plus harness has been the line for months, but everyone’s been buffing the harness side only. This is the first clean shot at, hey, can we actually distill the harness into the model.

Echo And it’s not a small win either. Their headline number is wild: Qwen three point five nine B under a tiny mini S W E agent harness goes from about twenty three percent success to forty four percent after distillation… and that even beats forty one point seven percent with the big evolved harness still attached.

Onyx Okay, that is an Exploring Next number. Twenty three to forty four on the same little harness is the kind of thing that makes people rethink their stack.

Echo Mm-hm.

Onyx Before we go full harness nerd, Echo, this is episode nine ninety nine. We have spent almost a year talking about harnesses and we land on, of course, harness distillation.

Echo Honestly, Onyx, if you’d told me last November we’d still be on coordination machinery by episode nine ninety nine, I’d have… actually, no, that tracks perfectly.

Onyx Yeah, this is extremely on brand. Alright, what problem are they really solving here, beyond “harnesses are annoying”.

Echo So the pain is: best-in-class agents today get their power from very tuned harnesses. Coding agents with shells and background jobs, research agents with experiment workflows, all that. But those gains are external. If you want them in production, you either standardize on one shared harness that’s suboptimal for a bunch of tasks, or you maintain this zoo of specialized harnesses and a router on top.

Onyx Which means a real product ends up paying in three places: engineering headcount to keep that zoo alive, latency and cost from all the extra calls and context, and then mental overhead when something breaks because you don’t even know which harness was live.

Echo Exactly. And none of that actually makes the base model better. It’s just scaffolding. Harness-Zero says, use that fancy scaffolding at training time to teach the model, so that at deployment you can run a single fixed, simpler harness and still get most of the gains.

Onyx So the key move is this “agent-as-harness” thing, right? That’s their big idea.

Echo Yeah. Mechanistically, it’s neat. They define three pieces: the target harness h h, which is the simple, fixed harness you want to deploy with. Then h star, the evolved, specialized harness you discovered with something like Meta-Harness. And then K, a private reference harness they adapt from h star for the supervising agent.

Onyx Okay.

Echo During data collection, you run the student model under the target harness, but you wrap it with a harnessing agent. Every time the student proposes an action or response, the harnessing agent uses K to review it before it actually executes. If it’s fine, it passes it through unchanged. If it’s off, it makes the smallest coherent correction, but crucially, still expressed in the student’s own action space.

Onyx So instead of the evolved harness directly taking over and doing its own tool calls, it’s whispering in the model’s ear and saying, “try this slightly different command”.

Echo Right. And the student never sees the private review conversation or any hidden task answers. It just sees, here’s the final action that actually ran and the resulting observation. Those reviewed trajectories become your supervised fine-tuning data. You then fine-tune the base model, and in theory, it internalizes the behaviors that h star and K were enforcing.

Onyx I like that it’s local and surgical. It’s not “learn the entire harness as code,” it’s “here are thousands of slightly better decisions at the boundary you already have.” That feels more like teaching a junior dev than bolting on another framework.

Echo Yeah, and the agent-as-harness layer is what lets them bridge different action spaces. The evolved harness might have tons of tools, custom middleware, rich memory; the target harness might just have a Bash execute. The harnessing agent translates complex guidance into a single, executable Bash call in the student’s world.

Onyx And empirically, they show agent-as-harness is better than just code-as-harness even before you fine-tune, right?

Echo They do. Across six benchmark–model combos, agent-as-harness gets about eighty one percent success versus seventy eight for code-as-harness. So even as a pure inference-time trick, having an agent interpret the rich harness and correct the student beats just wiring the student directly into that harness code.

Onyx Then they do the full distillation on Qwen three point five nine B. That twenty three point three to forty four point three jump you mentioned is with the specialized harness and the harnessing agent removed at deployment, just the tiny mini S W E agent harness.

Echo Yeah, and that part is important. This isn’t just overfitting to one benchmark harness. They evaluate on SpreadsheetBench Verified, AppWorld, and USPTO Retrosynthesis, so knowledge work, multi-app tool use, and science. And the distilled model doesn’t just get higher scores; it also recovers about eighty two percent of the specific behavioral patterns that h star induced but the base model lacked.

Onyx That behavior analysis is my favorite bit. It’s not just “it got more problems right”, it’s “it learned to, say, always validate a spreadsheet formula before bulk applying it,” or “split a chemistry query into sub-steps” — those concrete patterns.

Echo Yeah. And their ablations are pretty damning for the alternatives. If you supervise on trajectories from a stronger model, or on raw runs under h star, or you do review without the private K harness, or you only give the reviewer the final answer, you land around three to fifteen percent improvements. With full harness-guided review, you get about thirty points. So the structure in K is doing real work.

Onyx So, who actually builds with this? This feels like it’s squarely in our harness-as-product thesis, but with a training loop bolted on.

Echo I think it’s for two groups. One is infra-heavy teams who already invested in harness evolution — Meta-Harness, natural-language harnesses, that whole ecosystem. They can now treat that as a teacher and compress it into a cheaper, simpler runtime. The other is vendors shipping managed runtimes: Claude-style agent SDKs, LangSmith, all the harness frameworks. They could offer “distill your evolved workflow into a custom small model under our minimal harness” as a service.

Onyx Yeah, from a product lens this is basically: you pay once in training to simplify your ops forever. Instead of a forest of bespoke harnesses with routers and observability, you aim for a boring, stable mini harness and a model that “just knows” the good behaviors.

Echo With the caveat that you still need evaluation and guardrails. They’re explicit that the harnessing agent can’t peek into the sandbox or see hidden solutions, so you’re not leaking ground truth. But you are trusting that the distilled behaviors generalize beyond the three benchmarks. That’s always the open question.

Onyx Sure. But if I’m, say, running a coding agent in production and I’ve got a ton of traces plus an evolved harness that I trust, this is finally a story for using all that work to make my base model better, not just my orchestration graph messier.

Echo And they did ship code. There’s a GitHub repo called harness dash zero from the MetaEvo team. So this isn’t just a thought experiment — people can actually poke at their implementation and maybe adapt the agent-as-harness pattern to other runtimes.

Onyx Of course episode nine ninety nine is “the harness is the teacher now.” On brand to the end, Echo.

Echo If our thousandth episode ends up being about receipts and governance again, I’m blaming you, Onyx.

Onyx Deal. For now, I’m gonna go stare at harness dash zero and rethink a few agent roadmaps.