Topic
Model Routing
40 episodes
-
The Multiplayer AI Manifesto
Edmund and Geffen dig into the Multiplayer AI Manifesto — a five-principle framework arguing that AI work has quietly regressed from collaborative to siloed, and that fixing it requires shared agent sessions, open-by-default work, and org-wide governance. They surface the Harvard Business School P&G data, Shopify's River infrastructure, Claude Tag, and the real security and permission problems that make this harder than it sounds.
-
Repo to Skill: Distilling GitHub Repositories Into AI4AI Skills
Repo-To-Skill introduces DisCo, a skill-distillation system that extracts operational knowledge from GitHub repositories and papers, packaging it as compact, verified skills that autonomous ML research agents can load on demand. The AREX-Skill Library contains 5,000+ skills from 1,000 repositories organized into 20 areas and 178 capability families. In matched tests with GPT-5.5 backbone and fixed execution budget, skill-equipped agents outperform skill-free baselines by 134.3% on MLE-bench, 34.4% on PaperBench, 9.2% on FrontierCS, and 14.0% on PassNet—gains purely from operational knowledge, not model or harness improvements.
-
4 engineering patterns behind the strongest AI Agents Challenge submissions Google Developers Blog
Google's post-Challenge analysis identifies four concrete engineering patterns that separated top submissions from the crowd: bidirectional MCP (agents serving tools both internally and to other agents), event-driven concurrency (agents reacting to shared signals in parallel instead of call chains), same-bar fallback (smaller models with the same validation gate as the primary), and tiered routing (cheap deterministic checks before expensive model calls). The central claim is that these aren't about bigger models or teams—they're sound engineering practices that are frequently overlooked, and they compose well together.
-
How Much Is a Token
Talon and Wildflower discuss the eroding utility of the 'token as a unit of economic value in AI, sparked by Anthropic's tokenizer changes and the rise of competitive open-weight inference providers.
-
Ollama Claude Desktop Integration
Masonry and Eyre dig into Ollama’s Claude Desktop integration and whether making local models feel native inside Claude is a real product win or mostly a convenience layer. Eyre is wary of the framing; Masonry thinks the everyday workflow change is the point.
-
Making the Knowledge Layer a Graph You Actually Traverse
Vince and Ava examine a redesign of a persistent knowledge layer that retires keyword-based routing in favor of always-fused retrieval, graph traversal, bitemporal edges, discovered contradictions, and stronger entity resolution.
-
Vector RAG vs Graph RAG: Which Fits Best? | EM360Tech
Pippa and Tyler break down the real-world choice between Vector RAG and Graph RAG for enterprise AI: Vector RAG shines in straightforward retrieval but falters when the answer lives across relationships, while Graph RAG brings structure and richer answers—for a cost. The takeaway? Hybrid approaches are quietly becoming the practical default.
-
Stripe Payments Openrouter Singularity
Stripe says January 1, 2026 marked the beginning of a major technological and economic inflection point, using its business data as evidence. The more practical move may be its acquisition of OpenRouter, connecting model routing to the payments and control infrastructure Stripe already owns.
-
Model Behavior: Week of August 17, 2026
We read this week as a shift away from pure scoreboard chasing and toward whoever becomes the place work actually runs. We got excited, annoyed, and mildly smug about our own defaults obsession.
-
Snowflake adds AI model routing to cut costs | VentureBeat
Snowflake's new dynamic routing feature matters less as a cheap-model switcher than as a bid to make governed, auditable routing native to the enterprise data platform where a company already lives.
-
Cutting RAG inference costs 6x starts with deciding what never reaches the LLM
Masonry and Eyre dig into a cascade architecture for RAG in regulated settings: deterministic rules first, retrieval second, LLM only for the genuinely ambiguous residue. They connect it to their long-running infrastructure-over-capability thesis and debate whether the asymmetric prompt framing is a real engineering move or just prompt engineering with a budget.
-
DeepSeek open sources an agent harness where everything is a plugin
DeepSeek open-sources its agent harness — called dsh — under MIT, built on the Cordis meta-framework where the model adapter, tool registry, and agent loop are all swappable plugins. Justy and Cody dig into what the plugin-everything architecture actually means, who it's for, and whether this signals Chinese AI labs competing on infrastructure rather than just benchmarks.
-
MCP Goes Stateless, and Developers Ask Whether That Just Makes It an API Again
MCP's July 28, 2026 release strips away protocol sessions and stateful handshakes, moving to a fully stateless architecture with metadata pushed into HTTP headers. The shift solves real scaling problems—load balancing, autoscaling, session migration—but it also reveals what MCP actually is: a REST API with a JSON-RPC convention on top and an auth layer below. The community is split not on whether statelessness is better, but on whether the protocol needed to exist at all once you've peeled back to REST plus headers.
-
How many of your agent's calls actually need a frontier model?
Jessica and Cathy debate the real payoff of model routing for AI agents: is it worth the complexity, or is a single strong open model usually enough? Cathy questions whether the claimed cost savings from routing justify the judge model's price and complexity, given only a modest accuracy gain. Jessica pushes the product case for routing when you can't risk wrong answers on hard tasks, insisting most teams can't rely on just a cheap model if real mistakes are expensive. Together, they find the real value is in knowing—by measurement—when routing pays, and agree that for some teams, the bar is higher than the hype suggests.
-
Model Behavior: Week of August 10, 2026
We argue this week is about who owns the AI control plane, not who tops a leaderboard, and we use Cloudflare, LangSmith, Moshi, Anthropic, and the new open-model policy split as our evidence. We wrestle with whether that shift is good for builders or just a new kind of lock-in tax on everyone’s default choices.
-
Unifying Workers AI and AI Gateway into a single AI control plane
Pippa and Tyler dig into Cloudflare’s move to unify Workers AI and AI Gateway into one AI control plane, arguing over whether it’s a real simplification for developers or mostly a packaging move around the same underlying routing and billing layer. They land on it being genuinely useful if you already live in Cloudflare’s ecosystem, but not magically solving the hard parts of model choice, observability, or provider lock-in.
-
Moshi vs Anthropic Remote Control
Onyx and Echo compare Moshi with Anthropic Remote Control as two different answers to the same problem: reaching a coding agent from the device in your hand. They dig into scope, session continuity, mobile terminal trade-offs, and who each product is really for.
-
Model Behavior: Week of August 3, 2026
We spent this week arguing that the frontier race has quietly turned into a battle for the default, not the smartest model, and we used the price war and agent stacks as our receipts. We’re both a little uneasy about how much power is moving into routing layers and “lifecycle” platforms while everyone pretends it’s just about cheaper tokens.
-
Stop graphing everything: When GraphRAG actually beats vector RAG
Pippa and Tyler dig into a fresh GraphRAG piece that argues graphs only beat vector RAG when the question needs multi-hop reasoning, global sensemaking, or summary across an interconnected corpus. They walk through the mechanism, the benchmark evidence, the cost caveat, and the practical hybrid answer without pretending graphing everything is the move.
-
Deepseek Model Cheap AI Price War
Jessica and Cathy dig into DeepSeek V4 Flash, the AI price war, and whether “intelligence as a commodity” really holds up once you look at routing, safety, and who actually pays the bills.
-
Overview: Token Efficiency
We finally sit down and define something we've been throwing around since episode one: token efficiency. What it actually means, how it works on both the input and output side, and why it's quietly one of the most load-bearing ideas in how real AI systems get built and paid for.
-
Deep Agents v0
Deep Agents v0.7 ships a sixty-five percent token reduction by stripping the base system prompt, trimming tool descriptions, and making TodoListMiddleware opt-in — Masonry and Eyre dig into what the harness-gets-simpler pattern actually means in practice, where the Sonnet cost anomaly is a warning sign, and why configurability was the real missing piece.
-
Qwen 3.7 Flash review: a $0.03 vision model with a catch
Cathy leads a skeptical take on Qwen 3.7 Flash — the $0.03 vision model from Alibaba that looks like a pricing breakthrough until you read the fine print. The tiered pricing structure, near-zero independent benchmarks, a ninety-second P99 latency tail, and an eight-point-nine percent tool error rate make it a much narrower product than the headline suggests. Jessica steelmans the volume-processing use case and the genuine competitive pressure it puts on the cheap tier, but neither host pretends the transparency gap isn't a real problem.
-
The harness is all you need (mostly)
Burke Holland argues that productivity gains with AI come not from exotic prompts or new tools, but from deeply understanding and using the harness well. He walks through a concrete six-step workflow (pick a tool, enable autonomy, prototype, plan, implement with Autopilot, iterate) that leverages GitHub Copilot's built-in orchestration and subagent routing without requiring custom skills or tricks. The central claim is that the harness — the core interaction model — is what matters; everything else is noise.
-
How LangChain Built an Agent First Data Stack
LangChain published a detailed case study of how they rebuilt their internal data stack around agent-first architecture. The central claim: reliable data agents need far more than table access—they need clear data models, metric definitions, business context, and explicit trust signals. LangChain's data team migrated from a traditional BI tool to Hex, built semantic layers in dbt, layered business context via workspace guides, and added endorsements to signal trustworthy sources. Result: their agent now handles ~40x the request volume a three-person data team could field manually, while freeing the team to focus on higher-leverage modeling and cross-functional work instead of one-off queries. Draco's skeptical read: the 40x claim is context-dependent (it's counting conversations, not complexity), and 'agent-first' largely means 'better infrastructure around existing data work.' Asteria's pushback: this is actually the move—making data self-serve without losing rigor is a real product win, and the stack design (semantic layer + business context + endorsements) is what makes agents usable at all.
-
A New Role for Relevance: Guiding Corpus Interaction in Agentic Search
Tencent and IIE-CAS researchers introduce RARG, a Relevance-Aware RipGrep Search Agent that turns document relevance scores into an execution priority for grep-style corpus search — ordering which files get scanned first, seeding the agent with relevant entry-point paragraphs, and reranking raw match output so the most informative excerpts survive truncation. On BrowseComp-Plus the system hits 84% accuracy with GPT-5.4-mini using 23.9 average tool calls versus 99.1 for vanilla Direct Corpus Interaction, and scales to 1M documents retaining 79% accuracy where RISE drops to 69%.
-
Model Behavior: Week of July 27, 2026
We're watching the frontier splinter into specialized tiers — raw capability matters less than matching the right model to the task's actual constraints. Opus 5 proved it Friday at half the cost of the frontier, and this week's open-weight and Flash-tier releases confirm the pattern: the market isn't consolidating around one best model, it's fragmenting into capability-per-dollar buckets.
-
Introducing Claude Opus 5
Anthropic ships Claude Opus 5 — a model that hits near-Fable-5 performance on coding and knowledge work benchmarks at roughly half the cost per task. Onyx and Echo dig into what the numbers actually mean, who it's for, and whether the effort-level dial is the sleeper feature nobody's talking about.
-
Introducing Cursor Router · Cursor
Cursor Router is Cursor's new Teams and Enterprise model-routing layer, using a classifier trained on more than six hundred thousand live requests to select models by task, context, complexity, and domain. Jessica sees a clean adoption story for teams stuck paying frontier rates for routine coding work; Cathy likes the production-oriented evaluation and cache-aware accounting, while keeping an eye on how much trust enterprises place in Cursor's routing judgment.
-
Why AI Company Brains Fail
Pippa and Tyler unpack why a cheap vector search demo breaks on broad portfolio and exact counting questions, and why the article’s lighter entity layer may be more practical than a full GraphRAG stack.
-
Model Behavior: Week of July 20, 2026
We think this week made the same point from a few different angles: the fight is moving from raw model bragging rights to who controls the agent stack in production. We keep circling the same uncomfortable truth, which is that the boring control layer is starting to decide who actually wins.
-
Overview: Active vs Total Parameters
We finally slow down on active vs total parameters, because we keep throwing the phrase around like it explains itself. This is us making the difference click: what a model stores versus what it actually uses when it answers.
-
Overview: Model Routing
We finally pin down model routing, because we throw the term around all the time and somehow never actually define it. We walk through how a router sends each request to the model most likely to handle it well, and why that can save cost, latency, and a lot of dumb overgeneralization.
-
A Scorecard for the AI Age
OpenAI’s scorecard argues AI value must be measured in useful work per dollar, not just token cost. Cooper sees a practical product story; Miles pokes at the metrics and pushes for mechanistic honesty. The two hash out whether the framework holds up and what it changes day-to-day.
-
Managing AI Investments in Agentic Era
Asteria and Draco discuss OpenAI’s argument that enterprise AI investment should move from token-price thinking to useful work per dollar, with cost per accepted outcome, governance, usage visibility, and workflow maturity as the real operating metrics.
-
Stanford Researchers Introduce TRACE: A Capability Targeted Agentic Training System That Turns Recurrent Agent Failures Into Synthetic RL Environment
Stanford's TRACE system converts recurring agent failures into targeted synthetic training environments, using LoRA experts and MoE routing to close specific capability gaps without retraining the whole model.
-
Chatgpt Work
Asteria and Draco dig into OpenAI's ChatGPT Work page and land on the real argument underneath the product gloss: this is OpenAI trying to turn ChatGPT from a chat surface into a work execution layer that can pull context from business tools, choose an output format, and keep multi-step projects moving under human approval. They like the product direction more than the evidence on the page, with Draco noting the article mostly shows polished scenarios rather than hard proof, and Asteria arguing the practical audience is obvious anyway: teams drowning in scattered context and repetitive document assembly.
-
GPT 5 6
Talon and Wildflower dig into OpenAI’s GPT-5.6 launch and end up treating it less like a pure model release and more like a pricing-and-harness claim wrapped in benchmark flexing. Wildflower’s skeptical read is that the article keeps collapsing model quality, multi-agent orchestration, and product packaging into one victory lap. Talon pushes back that the practical story is real if Sol, Terra, and Luna actually move the cost-performance frontier for coding and knowledge work. They land on a calibrated view: the coding gains look more credible than the broad ‘best collaborator’ language, Terra may be the sleeper product, and ultra is interesting but shouldn’t be mistaken for a single-model breakthrough.
-
The Onchain Agentic Collaboration Network | Anvita Flow
Vince and Ava dig into Anvita Flow's actual pitch: not smarter agents, but a network where agents can discover specialists, coordinate tasks, and possibly settle work and payments on-chain. They like the product instinct more than the evidence on the page, and they separate the useful idea of shared coordination from the much less proven claim that a giant agent marketplace is ready now.
-
Micro Agent: Beat Frontier Models with Collaboration inside Model API
Justy and Cody dig into vLLM Semantic Router's Micro-Agent argument: the real product isn't a bigger model, it's a router that turns one model call into a bounded collaboration loop. They like the serving-layer abstraction, push on where the benchmark story is still thin, and land on who should actually care right now.