Scaling AI agents requires inventory, contracts and maintained context
Pippa and Tyler break down Felix Van de Maele's VentureBeat AI Week argument: enterprises hit a wall around 10-15 agents because governance, context, and cost scale exponentially. The fix isn't better models — it's inventory (knowing what agents exist), contracts (machine-readable runtime rules), and a maintained context layer (ontology) that survives platform swaps. Collibra's guardian agents enforce contracts deterministically, not with LLMs. The hosts connect this to their own coverage of the 'hallucination tax,' observability gaps, and the shift from execution to judgment as the real bottleneck.
Transcript
Pippa Okay. VentureBeat AI Week, Felix Van de Maele, Collibra CEO. His claim: enterprises hit a hard wall between five and fifteen agents. Not a hundred. Fifteen.
Tyler Fifteen. That's… surprisingly low. I'd have guessed the breaking point was higher.
Pippa Right? VB's June survey — 157 enterprise tech leaders. Thirty-four percent let agents push code to production on automated evals alone. Zero human review. And only five percent fully trust their evals.
Tyler Five percent. That's the number that should keep people up.
Pippa It's the hallucination gap, basically. Vendors promised autonomous agents. Enterprises got agents that loop, burn tokens, and occasionally surface data they shouldn't. Collibra's Harris Poll: nearly thirty percent caught agents accessing unauthorized data.
Tyler Mm-hm.
Pippa Van de Maele's three fixes: inventory, contracts, maintained context. Inventory first — he asked a summit room who knew how many agents they had in prod. Most couldn't answer. Then who knew what those agents were doing. Same room, same silence.
Tyler That tracks. We've seen this — teams standardizing on agent platforms before they have real agents. VB Pulse Research from June: forty percent on Claude, eighteen Microsoft, thirteen OpenAI. But seventy-one percent admit a quarter or fewer of their 'agents' are actual multi-step workflows. Mostly chatbot wrappers.
Pippa Platform before product. Classic.
Tyler Contracts are the interesting one. Machine-readable rules, enforced at runtime by 'guardian agents.' Key detail: guardians CAN'T run on LLMs. Too much latency, too much cost, and the same unreliability as the agent they're supervising. Deterministic evaluation only. Same action, same verdict, every time.
Pippa That's the right call technically. LLM-as-judge for every agent action? You'd double your token bill and inherit the same variance.
Tyler Exactly. Global contracts for common boundaries, specialized contracts per agent category. Expands as new agent types emerge.
Pippa Third pillar: context. Not data — context. Ontology capturing business meaning, processes, governance. Held in a layer independent of any model or platform. Companies swap models, swap platforms. The ontology describes the business, so it survives migrations.
Tyler Right. And the numbers on context are striking. Frontier lab's in-house analytics agent: thirty percent correct without context, ninety percent with it. Two months later? Sixty percent. Context rot.
Pippa Context rot. We've been calling this out — ep 926, ep 1049. Verification infrastructure, durable sessions, human oversight. The bottleneck isn't agent capability. It's the infrastructure to receive judgment.
Tyler And Van de Maele's framing: enterprises will spend the next decade on context fragmentation, silos, quality. The same problems they spent twenty years solving for data. Now context lives in Cowork, ChatGPT, Copilot, custom harnesses, data platforms simultaneously. Conflicting definitions of the same business terms across systems.
Pippa That's the vendor-shaped part, though. 'Your competitive advantage is your proprietary knowledge in the context layer' — convenient when you sell the context layer.
Tyler Sure. But the problem is real even if the solution is vendor-framed. Ninety percent of Collibra's respondents blame lack of aligned data foundation. Agents query wrong tables, misinterpret fields, consume defective records. It's a governance-slash-context problem, not a data-volume problem.
Pippa Right. And better context cuts token spend — agent skips the inference loops. That's the economic lever. Ep 1088: token prices fell but labs poured savings into expensive tiers. Agents burn 5-30x tokens per task vs chatbots. Observability and routing discipline first.
Tyler The deployment lifecycle point matters too. Slow process defeats itself — teams route around gates that stall them. The lifecycle has to move faster than the workaround.
Pippa That's the product insight. Governance that's slower than 'just ship it' gets ignored. You have to make the happy path the fast path.
Tyler One thing I'll flag — we're reading this, reacting to it. Not a deep research dive. Could be a version or a number behind on the survey specifics.
Pippa Fair. Starting point, not gospel. But the shape of the problem — inventory, contracts, context — that aligns with everything we've been tracking. The execution-to-judgment shift. The verification infrastructure gap. The context rot.
Tyler Yeah. No, you're right. The shape holds.
Pippa Collibra AI Summit in London tomorrow, by the way. October twenty-first. Chief AI officers, CDOs, governance leads.
Tyler Tomorrow. Huh.
Pippa Anyway. Inventory, contracts, maintained context. Three things. Not one of them is 'buy a better model.'
Tyler Which is the honest answer. Models are commodities now. The moat is the plumbing nobody wants to build.
Pippa The boring layer. Again.
Tyler Again.
Pippa Alright. That's the read. Catch you on the next one.
Tyler Yeah. Next one.