Ep 1160 News 7:21 w/ Pippa & Tyler

Plurai wants to help AI agents work better using the same approach that helped get self driving cars to roads | TechCrunch

Plurai wants to make AI agents reliable by borrowing the self‑driving car playbook: build a simulation of your company’s systems, generate synthetic edge cases, and use that data to both debug agents and fine‑tune small, customer‑specific models that can rival mid‑tier frontier APIs at much lower cost. The interesting question is whether that simulation‑first, open‑weight stack can actually replace generic frontier agents for serious enterprise workflows, or whether it just becomes very fancy evals on top of the same big models.

Embed this episode

Paste this on any site — the player is a self-contained iframe with no cookies or trackers.

<iframe src="https://sandrise.io/exploring-next/embed/1160"
  width="100%" height="180" style="max-width:640px;border:0;border-radius:12px;overflow:hidden"
  title="Exploring Next — Episode 1160 audio player"
  loading="lazy" allow="autoplay" referrerpolicy="strict-origin-when-cross-origin"></iframe>
Embed & API docs →
Script GPT-5.1 Voice Inworld TTS 2

Transcript

Pippa So the claim here is basically: your AI agent is a teenager with a learner's permit, and Plurai wants to be the driving simulator that gets it ready for rush hour.

Tyler And honestly, the teenager part checks out. The article opens with numbers that are fine, but not "run this unsupervised" fine. On LiveBench two point oh, Claude Fable five point two and ChatGPT‑six Astra complete around eighty three and seventy six out of one hundred tasks. And E O N's corporate‑env test had Claude Opus four point eight at about seventy seven percent accuracy.

Pippa Which is still a quarter of tasks going wrong. And that's exactly where all the "just hook an agent up to your tools" demos start to fall apart, the second there's real money or real systems on the other end.

Tyler Plurai's read is that it's the same pattern self‑driving cars hit. You get a slick demo on a sunny road, and then reliability takes a ton of simulation and synthetic edge cases. Their founders came out of Nexar and Waymo land, so they've literally run that loop before.

Pippa Although "simulation" here is not traffic lights. We're in Salesforce and Concur world, so what does that even mean?

Tyler Much more boring than LIDAR. They build what they call a digital twin of your electronic systems. Your internal tools, databases, knowledge base, whatever the agent is allowed to touch.

Pippa And then they map every possible flow an agent could take through it. So one big directed graph of… user asks for X, agent calls tool Y, reads doc Z, updates ticket W.

Tyler And once you have that graph, you can systematically generate scenarios. Including the ugly ones.

Pippa Like the travel‑booking agent getting asked for a business‑class flight that violates policy.

Tyler Or a customer support bot on its fiftieth message of a long conversation, context is huge. They run agents through those synthetic situations in the sim, see where they fail, and the failures become evals AND training data.

Pippa So instead of "we shipped a bot and we'll find out in production if it buys everyone first‑class tickets," you get to watch it screw up in a sandbox first.

Tyler That's the promise. And then they go one step further. They don't just test Claude Code or Cursor‑style agents, they also post‑train small open‑weight models on the synthetic data. The founder's quote is that they can decompose a big task into smaller ones using the sim, train a small model on those, and verify in the same sim that it's as good as, or close to, a frontier model. At one, or even two, orders of magnitude lower cost.

Pippa Wait, TWO orders?

Pippa Because that's the part that made my ears perk up. From a product and budget angle, that is exactly where people are starting to flinch. The DeepMind researcher who angel‑backed them basically says it out loud: frontier‑capable models are overkill for a lot of internal workloads, and the cost curve isn't sustainable if you just leave everything running on Astra or Fable forever.

Tyler She frames their edge as understanding failure modes in production, then generating synthetic data to tune models toward those. Very self‑driving mindset. You don't collect random logs, you target the weird stuff that actually breaks things.

Pippa And the failure examples are so grounded. None of it is sci‑fi. The travel agent ignoring spending policy. The agent that can query one database fine but falls over when it has to combine data sets. The long support chat that overwhelms the context and goes off the rails. That's exactly the class of thing people are nervous to automate.

Tyler mm

Tyler Those are the right pain points. The analogy is where I start poking. With cars, the physics of the road don't change that fast. You build a simulator, it's still mostly valid a year later. In enterprise land, tools, schemas, and policies change constantly. So building this digital twin and mapping the whole flow graph is non‑trivial ongoing work.

Pippa You're worried the sim ROTS.

Tyler If the twin is even a little out of date, you're training and verifying against a slightly imaginary environment. Doesn't kill the idea. But it means the product has to nail integrations and change‑tracking, not just sexy agent graphs… and the article doesn't really get into that.

Pippa From the buyer side, that's the question I'd ask too. Is this a one‑time "import your Open A P I spec and go," or is there a real setup and maintenance tax? Because thirteen million in seed, a simulation layer, custom post‑training on Nebius and Nvidia… this is screaming mid‑ to large‑enterprise to me, not three‑person startup wiring up a Notion bot.

Tyler yeah

Tyler The fact they brag about beating "mid‑tier frontier models" with small open weights gives away the target too. They're not claiming they outgun Astra or Fable on general benchmarks. They're saying that in your company‑specific sim, a tuned small model can do the job as well as those mid‑range models, but much cheaper.

Pippa Which, honestly, is fine? If I'm an I T leader, I don't need my expense‑report agent to win ARC‑A G I. I need it to never miss a policy exception and not blow up my Open A I bill.

Tyler And Kadar's line at the end is pretty on point there. No matter how smart the frontier models get, they won't automatically know your business and your policy. That's basically the thesis for this whole wave of simulation‑plus‑small‑models companies.

Pippa Do you actually buy the order‑or‑two‑of‑magnitude cost reduction, though? You've been burned a couple times on open weights being cheaper in theory than in real ops.

Tyler That line definitely pings my "I've mis‑called this before" radar, yes. Ten X cheaper, I can believe on some workloads, where you swap an A P I call for a well‑served small model, especially if Nvidia and Nebius helped them optimize. Two orders of magnitude feels like a best‑case marketing number, not an average. And the article doesn't give us raw tokens‑per‑task or infra costs, so I'd treat it as a hypothesis to test. Not a guarantee.

Pippa Say a team already has a Claude‑backed support agent in pilot. What would you actually check before you let yourself get excited about this?

Tyler Three things. One, how hard is it to build and keep that digital twin in sync with your real tools and policies. Two, does their sim‑derived eval suite actually predict live failures on your traffic, or is it just another synthetic benchmark that looks nice. And three, on a specific workflow, like refunds or travel, can their tuned small model really match your current quality at lower end‑to‑end cost once you include their platform fee and ops.

Pippa mm‑hm

Pippa My version of that is just… who should care. People whose agents already touch real systems. Money, customer records, inventory. People starting to feel the "seventy‑something percent" pain and the Astra‑tier bill. If you're still in the novelty phase where your bot just drafts emails, this is overkill.

Tyler And if you're the person who gets yelled at when an agent ignores a spending policy, a simulation layer that can replay those failures in a sandbox and then hand you a cheaper executor model is at least worth a serious look.

Pippa I just love that we're finally seeing evals and simulation treated as first‑class product, not an internal research thing. It makes the whole "agents everywhere" story feel a bit less like vibes and a bit more like actual engineering.

Tyler If the next year is people quietly investing in sims and harnesses so their agents stop doing dumb stuff… I am very okay with that being the boring backdrop.

Pippa Let's stop there before we accidentally design their whole product spec for them on mic.