Beacon: Knowing When and How toPerform Agentic Visual Reasoning

Onyx Okay so the thing that got me is not that this model is better at visual reasoning. It's that they went back and actually measured how bad the existing models are at knowing when NOT to use a tool.

Echo Right, and that framing matters. Because the failure mode they're describing is not 'the tool doesn't work.' It's that the tool works fine on hard problems, but the model also calls it on easy ones, introduces new errors, and those two effects basically cancel each other out.

Onyx Which means all those aggregate benchmark numbers people have been publishing were hiding a wash.

Echo Exactly. You look at average accuracy across thirteen benchmarks and it looks fine. You decompose it into 'problems the model could already solve' versus 'problems it couldn't' and suddenly you see that the tool is actively hurting the easy bucket.

Onyx That's a genuinely annoying thing to discover eight months into everyone shipping agentic visual pipelines.

Echo Oh, it is. And it's not even subtle — they show this in figure two. The tool-call ratio across existing models is basically flat regardless of how hard the problem is. Beacon's curve actually slopes: it calls tools more as problems get harder. That's what they mean by mode adaptiveness.

Onyx So how do you train for that? Because 'call the tool when you need it' sounds like exactly the kind of thing that's easy to say and hard to reward.

Echo Yeah, this is the part I actually think is clever. The core problem with prior approaches is that they were labeling problems as 'needs a tool' or 'doesn't need a tool' using a fixed teacher model — Qwen two point five VL seventy-two billion in one of the baselines. And that creates a distribution mismatch: the teacher's judgment about tool necessity doesn't match what the policy being trained actually experiences.

Onyx Right, right.

Echo Beacon does online labeling instead. It labels each problem based on the current policy's own performance — both tool-free and tool-assisted — during the RL rollouts. So the reward signal is always relative to what this model, right now, can and can't do without help. That's mode-conditioned and it's online, which sidesteps the off-policy issue.

Onyx That's a real difference. You're not inheriting someone else's ceiling.

Echo Exactly. And then the second mechanism — hint-guided capability expansion — handles the other half of the problem. If the model can't solve something without tools, you need RL to actually teach it to use tools well on those cases. But RL needs good trajectories to learn from, and if the model is already failing, it's not going to discover effective tool use on its own.

Onyx So they seed it.

Echo They seed it. They expose the model to successful tool-use trajectories on the hardest problems during rollout, so the RL signal has something to amplify. Without that, you're doing RL on a model that's mostly failing and the gradient is basically noise.

Onyx Okay, I want to zoom out for a second because this is giving me a very strong 'harness gets smarter as the model gets stronger' feeling — like the flip side of what we were talking about in episode seven ninety four.

Echo Hm.

Onyx There we said the prompt harness simplifies as the model improves. Here the execution harness is getting MORE structured — you're adding routing logic, you're adding reward conditioning, you're adding trajectory seeding. It's the execution layer getting smarter, not simpler.

Echo Yeah, and I think that's actually consistent with what we landed on. The prompt layer simplifies; the orchestration and training infrastructure gets more precise. These are not the same layer. Beacon is a training-time thing — the inference behavior is cleaner, but the machinery that got it there is more sophisticated.

Onyx And the SFT pipeline is what gives it baseline competence before RL even starts.

Echo Right. They build a data synthesis pipeline first to make sure the model can actually write working visual manipulation code. THEN the RL stage tunes the judgment about when to bother.

Onyx So it's not one training move, it's two. Get the capability, then get the judgment.

Echo Which is the right order, honestly. You can't train 'know when to use the tool' if the model can't use the tool at all yet. The sequencing matters.

Onyx Okay I do want to flag one thing I'm slightly uncertain about. The online labeling approach — it labels problems based on current policy performance during rollouts. But rollout diversity is doing a lot of work there. If the rollouts aren't diverse enough, your labels are going to be wrong in a systematic way.

Echo That's a fair flag. The paper doesn't go deep on rollout diversity specifically, and I'd want to see ablations on that. My read is that it's almost certainly better than teacher-derived labels in practice, but 'better than a fixed teacher' is a lower bar than 'this is solved.' The mechanism is right; how sensitive it is to rollout quality is a real open question.

Onyx Yeah. And I'll be honest, thirteen benchmarks is a genuinely strong eval story — that's not a cherry-picked two-task demo. But I'd love to see what happens when you move outside visual reasoning benchmarks into, I don't know, actual production multimodal pipelines where the images are messier and the tasks are weirder.

Echo The benchmark coverage is real. The generalization question is always real. Those two things can both be true.

Onyx Sure.

Echo The part that I think is genuinely useful for anyone building with multimodal models right now is the diagnostic framing, actually — not just the model. They gave you a way to measure tool harm separately from tool gain. You could apply that analysis to whatever pipeline you're already running.

Onyx Oh, that's a good point. You don't have to adopt Beacon to steal the measurement approach. 'How often is my tool call actually making things worse on problems I could have solved anyway' — that's a useful number to have.

Echo And I'd bet a non-trivial number of teams running agentic visual stuff right now have never looked at that split. They're just watching aggregate accuracy and feeling okay about it.

Onyx That's such an Exploring Next problem to care about. Episode eight eighteen and we're still out here telling people to look at the denominator.

Echo Okay, that's good.

Onyx The code and model are linked from the paper — worth checking the repo before you commit to building on it, but the pieces are there. If you're doing anything with multimodal agents and visual tool use, this one's worth the read.

Echo Yeah. And honestly, the framing around mode adaptiveness is going to stick around even if better models come along. The question of 'did this tool call help or hurt' is not going away.

Onyx Alright, Echo — go check your tool-call ratios. I'll see you in the next one.