AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scale

Justy This is the version of agent training where somebody finally admits the task isn't the whole world.

Cody Right. AgentMercury is trying to build the world first, then let tasks fall out of it, which is a much saner framing for business workflows than another tiny benchmark maze.

Justy And honestly, my week has been a little full of papers calling a prompt template an environment. So this was weirdly refreshing. They actually give the agent state, services, tools, and consequences that persist.

Cody Their name for the world-builder is the Planet role. Slightly grandiose, obviously. But mechanically it has a real job: take a high-level business scenario and instantiate entities, service interfaces, state tables, transition dynamics, and constraints across services.

Justy Mm-hm.

Cody Then the policy acts in that world, while a separate world model can predict what happens after actions. The paper's useful distinction is that most work trains the policy or improves the world model, but treats environment construction as somebody else's manual labor.

Justy Which is exactly who has been stuck here: teams that need agents for ongoing operations, not one clean task with an answer key. A real workflow has an account record changing in one system, an approval in another, and some ugly rule that has to remain true across both.

Cody Exactly.

Justy You don't want to train on a task that says, "update the customer," and quietly assumes every downstream service behaved. You want the world to complain when the billing status and support entitlement now disagree. That is the part a product team eventually gets paged over.

Cody And AgentMercury's answer is executable invariants. Instead of hard-coding a task's success path into every transition rule, it writes world-level conditions as deterministic SQL-based verifiers. The agent can take different routes through the same workflow, but the verifier checks whether the cross-service state still makes sense.

Justy Right, right.

Cody That separation matters. World construction is one layer, task instantiation is another, and interaction comes after. So one persistent environment can support different objectives without rebuilding the whole simulator around each new benchmark.

Justy This is our receipts thing again, Cody. Every agent company discovers checks and logs, paints them metallic, and calls it an operating system by lunch. But here the boring receipt is genuinely the enabling move.

Cody Yeah, because a verifier gives reinforcement learning a stable signal. The paper built four thousand seven hundred eighty-three executable environments across fourteen industries and fifty countries, then trained policies inside those worlds without targeting the evaluation benchmarks.

Justy And the scale is what makes me sit up. A business-scenario generator could be useful for companies that have plenty of workflow documentation but cannot afford to hand-author every training environment. Though I would not call this plug-and-play production software yet. It feels closer to a serious training substrate than something a support-ops team installs on Tuesday.

Cody Sure.

Justy Also, can we acknowledge that "Planet role" sounds like a job title invented by someone who refuses to update a spreadsheet? "Sorry, I can't approve that expense. I'm responsible for terrestrial state transitions."

Cody It does sound like a very expensive badge. But the authoring result is substantial: they fine-tuned Qwen three point five, thirty-five billion A three B, on construction traces, and held-out executable-world authoring success rose from three point three percent to eighty-three point three percent.

Cody That suggests the construction process itself is learnable, not just a one-off prompting trick. Still, I'd want to inspect what counts as authoring success. Does the generated world merely execute, or does it have enough meaningful state and constraints that agents cannot game it with shallow behavior?

Justy That is fair. The headline gains are also big enough that I want the annoying details. Qwen three point five, four billion, goes from twelve point three to fifteen point seven on EnterpriseOps-GYM, and from forty-five point nine to fifty-six point zero on AIME twenty-six. I buy that richer interaction can transfer, but that AIME jump needs careful controls.

Cody I see.

Cody Especially because broad transfer is easy to overread. Their environments were not built around those evaluations, which is the interesting claim, but we still need comparisons against equally large task-generated data, matched training compute, and leakage checks. Otherwise we cannot tell whether scenario grounding is the cause or just more useful rollout volume.

Justy Yeah, no, you're completely right. But if the effect holds under those controls, this changes the buyer conversation. You are no longer buying a benchmark pack. You are building a reusable little company-world where training tasks emerge from the rules you already care about.

Cody And that is more aligned with the long-running fight over whether agents need giant context or real durable state. This paper is firmly in the state-and-coordination camp, but it does not settle that fight. It gives that camp a much better way to create practice worlds.

Justy The paper says it releases the synthesized environments and tasks, construction code, and trained policy models through the AgentMercury Huggingface Project Page. I'd start there, then try one scenario your team can actually verify rather than generating ten thousand decorative planets.

Justy Okay, Planet manager. Put the verifier in the backlog before you name the universe. I cannot believe episode eight ninety-nine got us here, but this one earned it.