Jev in the Agent Loop: A Complete Guide to Decision Layer Automation
Justy and Cody dig into "Jev in the Agent Loop," arguing the real move isn’t smarter models, but cheaper, structured decision-making—replacing frontier-priced LLMs on every internal fork with Jev. They debate where Jev actually fits, what it can’t do, and why this reframes agent infrastructure as cost optimization at the decision layer, not generation. Cody pushes back on the limits: Jev can’t generate, so it only works in pipelines where you can cleanly separate choosing from doing. Justy sees immediate product wins in agent tooling, and they swap examples and a little banter on what gets hyped as "agentic" when it’s really just moving decisions around.
Transcript
Justy Okay, Cody, I want to drop you right at the heart of this. The article’s claim is literally: most of what agents do isn’t language—it’s making little forks all the way down, and that’s where the cost goes.
Cody Right. It breaks down every loop: which tool to use, who works next, is the task done. All those micro-decisions are usually handled by the same giant LLM, billed at LLM rates, with seconds of latency and the risk of a bad string breaking your pipeline at three in the morning.
Justy Which is insane, because none of those steps need generation. The trick here is Jev—you just send it typed questions, it sends back structured answers, no strings, probabilities included. Seventy to five hundred milliseconds, four cents per million tokens, output is free.
Cody Yeah, and it’s not pretending to be an agent. Jev can’t generate; it just chooses. The piece that jumped out for me is that Jev runs right next to real LLMs—Browser Use pairs Jev with a typing LLM, fraud detection runs Jev for the majority case and Kimi K3 for the weird tail. LangChain’s even pairing Jev with GPT-5.6 Luna and Sol in middleware now.
Justy Right—so instead of migrating off your main model, you just subtract the stuff it was never good at. The article is honest about it: Jev can’t be your planner or writer or coder. It’s the decider, not the doer.
Cody And the engineering shape is all about boundaries. Every Jev call is state plus a typed map of questions—you get back structured answers, with calibrated probabilities. Choice for routing, Score for grading, Noul for yes/no. You’re on the hook for thresholds, but the split is clear: Jev reports, your code decides how risky you want to be.
Justy Honestly, that’s a product win. If you've ever tried debugging an agent that hallucinated a label or spat out a malformed JSON—this is the opposite. Audit trails, explicit gates, all the boring admin-console stuff we keep saying is the real maturity layer.
Cody Mm-hm.
Justy I mean, remember our running theory from episode one thousand thirteen? Fix the plumbing, output just works. This is that—decision-layer plumbing.
Cody Exactly. The key evidence is how it’s used live. The browser agent case is great—they let Jev pick actions, then only call a heavy LLM when there’s actual text to type. That’s not just cost—it's a whole order of magnitude on latency. Their real numbers: one second to compact a Claude session from nearly a million tokens to eighty-six thousand, just by scoring relevance.
Justy Oh interesting.
Cody But, and here’s the catch—this only works if your workflow really separates decision from generation. If the boundary is blurry, or you need creative glue, Jev can’t touch it. The system has to be architected for this split.
Justy That’s fair. Though the article’s production cases are tight: five hundred emails classified in seconds, a fraud pipeline sorting a hundred emails in one and a half seconds, with the expensive model only catching the weird edge. Seven cents for the whole run. Those are numbers you can show to actual humans who approve budgets.
Cody And the pricing is genuinely wild—four cents per million input tokens, and you only pay for input, not output. That’s forty to four hundred times cheaper for decision-heavy traces, if their claims hold on your workload. I’ll give them that.
Justy Yeah, this is not a replacement for everything, but for teams actually shipping agents, it’s instantly interesting. The code is just a different decision model drop-in—no migration drama, just 'stop burning frontier tokens on stuff that’s basically a switch-case.'
Cody Sure.
Justy And you know people will still call the whole setup 'agentic,' even though this is literally moving branches out of the LLM and into a better if-statement.
Justy That is the classic Exploring Next take—eighty percent of what’s branded as 'AI agent infrastructure' is just making the world's fanciest router. But I’m not mad about it. If it cuts the bill and makes the logs readable, I’ll take it.
Cody You’re not wrong, Justy. Product pragmatism survives another loop.
Cody Hey, before you close—did you catch the bit about tuning thresholds with actual labeled examples instead of optimism? That’s the real marker that this layer is maturing. Nobody’s just hoping a prompt works; you have feedback loops wired in.
Justy Now you’re speaking my language. Okay, I’m calling it—you and I have spent almost a year on this show, and if we don’t get a 'boring but works' tattoo soon, someone’s going to start a rumor.
Justy Alright, go update your agent config and try not to burn frontier tokens on another routing loop tonight, Cody.