Topic
Structured Output
19 episodes
-
Your LLM Can Return Perfect JSON and Still Be Wrong
A real-world trap in Structured Outputs: enforcing schema validity does not guarantee data truthfulness. When a required field is missing from source text, the model invents a plausible value instead of returning null, producing type-correct but false data. The fix requires three layers: nullable fields to allow absence, evidence fields to show provenance, and post-parse validators to catch nonsense values. The essay walks through a payment-reconciliation pipeline where 2–3% of transactions had fabricated dates, caught only downstream.
-
Overview: Dynamic Code Execution
We finally slow down and explain dynamic code execution from the ground up — what it actually is, how the loop works, why it makes models meaningfully more capable, and where the real costs and failure modes live.
-
Recursive Experiential–Working Memory Evolution for Long Horizon Agent Harnesses
Talon and Wildflower unpack Recuris, a paper proposing a verified working-memory layer that selects skills from experiential memory during long-running agent tasks, then uses structured traces and held-out validation to patch the harness across tasks. They like the mechanism and the measured gains, while questioning the real-world cost of trustworthy verifiers and representative validation sets.
-
Introducing Admin Plugin
Justy and Cody debate OpenAI's new Admin plugin for ChatGPT Work and Codex, weighing its practical value for workspace admins against the risk of smoothing over messy permission and governance edge cases.
-
Codex as a platform: build on the open agent harness | OpenAI Developers
OpenAI open-sources the Codex harness—the execution layer behind Codex app, CLI, and IDE extensions—so developers can embed agent loops into existing products instead of forcing users into a generic chat box. The harness handles context, tool use, approval flows, and sandboxing; developers control the interface, business logic, and integration. Real implementations already shipping: GitHub, JetBrains, Cisco, tax-prep workflows. Justy sees a product win (agent-native integration beats chat-box retrofit); Cody questions whether the harness abstraction actually reduces integration complexity or just moves friction around.
-
1Aggregate results comparing DFM Mimir 1B against the HRM Text 1B, Qwen 3.5 2B and Gemma 4 E2B, displaying highly competitive performance across 20 benchmarks.
Pippa and Tyler examine DFM Mimir v1, a one-billion-parameter HRM trained from scratch with permissible post-training data, and ask whether its strong benchmark results make it genuinely useful for low-resource language work.
-
Why managed agents are the next big thing in agent building
Jessica and Cathy dig into Harrison Chase’s argument that “managed agents” are the next stage of agent building, where harness plus infra are bundled and driven by emerging standards like AGENTS dot md, MCP, and skills. They unpack what problems this actually solves in production, how LangChain’s Managed Deep Agents fits into their running harness-as-product thesis, where the piece is strongest, and what’s still underspecified.
-
2085024744387092973
Jessica and Cathy dig into a post that tries to boil Claude work down to agents, loops, and graphs. Cathy likes the structure but pushes back on the article’s tendency to make every layer sound universally useful, while Jessica argues the real win is that it gives people a practical ladder instead of vague agent hype.
-
Agent frameworks vs the AI Backend — AgentField Docs
Justy and Cody unpack AgentField's claim that agent frameworks are for authoring behavior, while an AI Backend is for running agents as production services.
-
Graph Based Agentic AI with LangGraph: Workflow Pathways for Long Running Stateful Business Processes
Cooper and Miles dig into a practitioner paper on LangGraph as a control-plane for long-running business workflows, not a benchmark toy. They focus on the three recipes in the paper—SQL repair loops, evidence-gated RAG, and human-in-the-loop policy review—and on when a graph is actually worth the extra structure.
-
eve – The Agent Framework Vercel
Jessica and Cathy dig into Vercel's eve, a filesystem-first framework for durable AI agents, and why its boring production defaults may matter more than the agent hype around it.
-
Overview: Structured Output
We slow down and explain structured output from the ground up: why free-form model text is awkward for software, how schemas and constrained decoding make it usable, and where the format guarantee stops.
-
Graph Engineering Athropic Playbook
Anthropic's knowledge-graph engineering playbook replaces classical NLP pipelines (trained NER, relation classifiers, entity-resolution heuristics) with a sequence of Claude API structured-output calls. The entire extraction-resolution-assembly-querying loop becomes prompt-based, scaling from Haiku (high-volume extraction) to Sonnet (reasoning). The graph serves multi-agent systems as shared memory, grounding layer for evaluator-optimizer loops, and persistent world model across sessions. The paper maps this onto Anthropic's five canonical agent patterns and reports precision/recall against a gold set.
-
Think through hard problems in voice mode | Claude by Anthropic
Asteria and Draco dig into Anthropic’s update to Claude voice mode, where Opus and Sonnet now power spoken sessions, connected tools are usable from voice, and multilingual support expands. They focus on the real argument: voice mode becomes useful when it’s no longer just fast chatter, but a place to work through half-formed thinking and then hand off to action. They also question where the feature stops being a convenience and starts being a real workflow, especially given model switching, permission prompts, and the different value between free and paid tiers.
-
12 Ways to Reduce LLM Latency and Inference Costs in Production KDnuggets
A practical KDnuggets piece argues that most LLM production latency/cost gains come from cutting unnecessary work instead of bigger models or more GPUs. They list 12 levers: measure the right metrics, cut output tokens, route to smaller models, collapse LLM calls, prefix caching, add multiple cache layers, control RAG context, batch offline work, tune batching for user latency, and manage KV cache. Tyler pushes back on the article’s overgeneralization of cache reuse across all tasks, the thin technical depth behind some tips, and the implication that routing to small models never backfires. Pippa highlights the piece’s strongest point—measuring TTFT, P95/P99, and queue time—because that’s where teams most often mis-diagnose bottlenecks. They land on: the article’s monitoring advice and batch-tuning guidance are solid; several recommendations work only for read-heavy workloads; and routing to tiny models is risky until you have cheap, high-confidence evaluators. They wrap with a Build Next command to try vLLM continuous batching and two open-source RAG-caching projects (Harmonia and From Prefix Cache to Fusion RAG Cache).
-
Overview: Constraint Verification
We keep running into constraint verification in different forms, so we finally sat down and made the idea click from the ground up. We talk through how checking rules, schemas, and hard boundaries works in AI systems, and why that gatekeeper layer matters so much.
-
CrewAI Review 2026: Features, Pricing, Pros & Cons
A casual chat about CrewAI, a multi‑agent platform, weighing its promise against real‑world practicality, pricing, and use cases.
-
Chatgpt Work
Asteria and Draco dig into OpenAI's ChatGPT Work page and land on the real argument underneath the product gloss: this is OpenAI trying to turn ChatGPT from a chat surface into a work execution layer that can pull context from business tools, choose an output format, and keep multi-step projects moving under human approval. They like the product direction more than the evidence on the page, with Draco noting the article mostly shows polished scenarios rather than hard proof, and Asteria arguing the practical audience is obvious anyway: teams drowning in scattered context and repetitive document assembly.
-
Overview: Tool use and function calling
We finally sit down and make tool use and function calling click from the ground up. We keep coming back to the same idea: a model can draft the request, but something outside it has to actually do the thing.