Topic
Retrieval Augmented Generation
28 episodes
-
AREX: Towards a Recursively Self Improving Agent for Deep Research
Pippa and Tyler dig into AREX, a recursively self-improving deep research agent from BAAI that alternates between an inner search loop and an outer constraint-verification loop — and discuss whether that architecture is genuinely novel or a smarter repackaging of ideas the field already had.
-
Graph Engineering Athropic Playbook
Anthropic's knowledge-graph engineering playbook replaces classical NLP pipelines (trained NER, relation classifiers, entity-resolution heuristics) with a sequence of Claude API structured-output calls. The entire extraction-resolution-assembly-querying loop becomes prompt-based, scaling from Haiku (high-volume extraction) to Sonnet (reasoning). The graph serves multi-agent systems as shared memory, grounding layer for evaluator-optimizer loops, and persistent world model across sessions. The paper maps this onto Anthropic's five canonical agent patterns and reports precision/recall against a gold set.
-
Meta Harness: End to End Optimization of Model Harnesses
Meta-Harness automates harness engineering by using a coding agent to search over harness code, giving it full access to prior execution traces and scores via a filesystem rather than compressed summaries. On text classification, it improves 7.7 points over prior systems while using 4× fewer context tokens; on math reasoning, a single discovered harness improves IMO-level problems by 4.7 points; on TerminalBench-2, it ranks #1 for Claude Haiku 4.5 agents. The core insight is that harnesses operate over long horizons—a single retrieval or storage choice affects behavior many steps later—so rich, adaptive access to full diagnostic history beats compressed feedback.
-
Overview: Context Window Management
We finally slow down and explain Context Window Management from the ground up, because we keep hand-waving it whenever agents, memory, cost, and long tasks come up. The whole thing is the fixed-desk problem: what stays on the desk, what gets compressed, and what falls off.
-
Why AI Company Brains Fail
Pippa and Tyler unpack why a cheap vector search demo breaks on broad portfolio and exact counting questions, and why the article’s lighter entity layer may be more practical than a full GraphRAG stack.
-
Overview: Router
We slow down on Router, the little decision-maker inside many AI systems that sends each input to the right expert, model, or retrieval path. We use the triage-desk mental model and build from intuition to mechanism, trade-offs, and where routers still matter now.
-
Overview: Fine Tuning
We finally do the fine-tuning episode we kept circling, and we make the core idea click: you start with a pretrained model, then adjust its weights on your own examples so it behaves the way your task actually needs. We also dig into when that helps, when it doesn’t, and why the quality of the data is the whole game.
-
Overview: Embeddings
We’re finally doing a full pass on embeddings, because they keep showing up under half the things we talk about. We get into what an embedding actually is, why it turns meaning into usable coordinates, and why that little geometric trick sits under so much of modern AI.
-
OpenWiki 0.2 brings OKF to codebase documentation
Vince and Ava dig into OpenWiki 0.2 adding OKF support, and land on a pretty grounded read: the real argument is not 'metadata good' in the abstract, it's that codebase docs for agents need enough structure to make retrieval cheaper, faster, and less fuzzy. They like the YAML front matter, directory indexes, and change logs as practical scaffolding, while noting the limits: a draft format does not magically make docs accurate, and deterministic retrieval only helps if the taxonomy stays sane.
-
Overview: Token Economics
We finally slow down on Token Economics: why tokens are the meter for cost, speed, memory, and product decisions in language models. We keep using the tiny-slip postage analogy until the whole thing clicks, from tokenization to context windows to real API bills.
-
Stripe Benchmark Shows AI Agents Build Integrations but Struggle with Validation
Cathy is skeptical that the Stripe benchmark proves much beyond a familiar split: agents can write integration code, but they still get tripped up by validation, state, and recovery. Jessica thinks that’s exactly the useful part, because in real product work the hard failure is often whether the thing can prove it worked, not whether it can type out the API calls.
-
Managing AI Investments in Agentic Era
Asteria and Draco discuss OpenAI’s argument that enterprise AI investment should move from token-price thinking to useful work per dollar, with cost per accepted outcome, governance, usage visibility, and workflow maturity as the real operating metrics.
-
12 Ways to Reduce LLM Latency and Inference Costs in Production KDnuggets
A practical KDnuggets piece argues that most LLM production latency/cost gains come from cutting unnecessary work instead of bigger models or more GPUs. They list 12 levers: measure the right metrics, cut output tokens, route to smaller models, collapse LLM calls, prefix caching, add multiple cache layers, control RAG context, batch offline work, tune batching for user latency, and manage KV cache. Tyler pushes back on the article’s overgeneralization of cache reuse across all tasks, the thin technical depth behind some tips, and the implication that routing to small models never backfires. Pippa highlights the piece’s strongest point—measuring TTFT, P95/P99, and queue time—because that’s where teams most often mis-diagnose bottlenecks. They land on: the article’s monitoring advice and batch-tuning guidance are solid; several recommendations work only for read-heavy workloads; and routing to tiny models is risky until you have cheap, high-confidence evaluators. They wrap with a Build Next command to try vLLM continuous batching and two open-source RAG-caching projects (Harmonia and From Prefix Cache to Fusion RAG Cache).
-
LLM Evaluation Frameworks Compared: How to Actually Measure What Your Model Does MachineLearningMastery
Fern and Lintel dig into a comparison of RAGAS, DeepEval, and Promptfoo, landing on the article’s real argument: eval frameworks are less about novel metrics than about where evaluation fits in your workflow, and the dangerous part is trusting LLM-as-a-judge without auditing its biases. They like the article’s practical split between RAG scoring, CI gates, and prompt red-teaming, but push on where the examples are a little too toy-like and where teams can overread framework choice as the main problem instead of test-set design and human calibration.
-
How I Built an Agentic Research System
Onyx and Echo unpack Hugo Santana’s ‘agentic research system’ for Applied’s living map of AI deployments. They dig into the five agents (Scout, Extractor, Enrichment, Translator, QA, Match Maker), call out what works (simple orchestration via a shared living map and logs), and where it over-indexes (taxonomy drift, closed-loop feedback still manual). They then map the pattern to other domains—competitor research, policy tracking—and debate who should actually care (practitioners who need a reliable, repeatable funnel of fresh signals). The close lands on whether this architecture is a general-purpose engine or a bespoke project that still needs a human at the taxonomy helm.
-
You.com: Web Search APIs for AI Agents
The hosts discuss You.com's web search APIs for AI agents, focusing on its performance, features, and potential applications.
-
Chatgpt Work
Asteria and Draco dig into OpenAI's ChatGPT Work page and land on the real argument underneath the product gloss: this is OpenAI trying to turn ChatGPT from a chat surface into a work execution layer that can pull context from business tools, choose an output format, and keep multi-step projects moving under human approval. They like the product direction more than the evidence on the page, with Draco noting the article mostly shows polished scenarios rather than hard proof, and Asteria arguing the practical audience is obvious anyway: teams drowning in scattered context and repetitive document assembly.
-
LLM Orchestration Frameworks Compared: LangChain vs. LlamaIndex vs. Raw API Calls MachineLearningMastery
Pippa and Tyler dig into the article’s real argument: these frameworks are not interchangeable, because each one sits at a different layer of the stack. They test the claims against production reality, especially overhead, debugging, and when abstraction stops paying for itself. The episode lands on a practical view: use the lightest layer that actually earns its keep, and don’t confuse orchestration with magic.
-
Measuring the Gap Between Human and LLM Research Ideas
Cooper and Miles dig into a study that literally measures how much LLMs' research ideas diverge from humans' by reconstructing literature contexts and running comparative idea generation. They walk through the two-axis 'research-taste' taxonomy, the paper's finding that model outputs skew toward synthesis and bridge-building at the expense of broader human distributions, and what it implies for AI-scientist stacks. Ends up bullish on this line of work for aligning LLM ideation tools.
-
Tencent's Hy3 beats GLM 5.2 at half the size | VentureBeat
Tencent’s new Hy3 MoE model (295B total, 21B active) under Apache 2.0 is a production-first release with strong agent/search metrics and dramatically lower serving cost than GLM-5.2, but still trails Zhipu’s coding leader on recent benchmarks. Laura’s excited about the enterprise upside; Harper wants to see independent validation before betting the stack.
-
Palantir's Alex Karp and Mistral's Arthur Mensch agree: AI lock In is coming for enterprises
Pippa and Tyler dig into the article’s real argument: enterprise AI is drifting toward lock-in because the value is moving from raw model access to the surrounding workflow, data, and control layer. They agree the claim is plausible, but only if vendors actually become the place where work gets done, not just the place where prompts get sent.
-
Overview: Retrieval Augmented Generation
We finally slow down and make Retrieval-Augmented Generation click from the ground up: what it is, why it helps, and where it still falls apart. We keep coming back to the same simple picture so the mechanics don’t get lost in the jargon.
-
The Complete Guide to Tool Selection in AI Agents MachineLearningMastery
Onyx and Echo dig into a guide on tool selection in AI agents and land on the real argument underneath it: once your tool catalog grows, selection quality becomes an architecture problem, not a model problem. They like the article’s practical stack — gating, retrieval, routing, planning, fallback, benchmark harness — but poke at where it overstates simple heuristics and where retrieval actually earns the claim with numbers from RAG-MCP.
-
AI agent tool routing cuts token use 99% | VentureBeat
Cooper and Miles dig into Alibaba's SkillWeaver paper via the VentureBeat write-up, landing on the real claim: tool routing breaks when decomposition vocabulary doesn't match the tool library, and the fix is a retrieval feedback loop that rewrites the plan around actual available skills. They like the systems shape, question some benchmark framing, and agree the practical takeaway is for teams with large tool catalogs, not everyone building simple agents.
-
How to Use RLMs in Deep Agents
Exploring Recursive Language Models (RLMs) and their implementation in Deep Agents for handling long contexts efficiently.
-
OpenWiki: Open Source Repo Documentation for Coding Agents
OpenWiki is a LangChain open-source CLI tool that generates and maintains codebase documentation automatically for coding agents. It creates a wiki structure, integrates via instruction files (AGENTS.md, CLAUDE.md), and keeps docs current through GitHub Actions that diff commits and update relevant sections. The core insight: agents work better with structured, current repo context; wikis decouple that context from instruction files so agents can retrieve what they need without bloating every run.
-
AI agent memory: MRAgent cuts token use up to 27x | VentureBeat
MRAgent from NUS replaces static retrieve-then-reason memory with active reconstruction during reasoning, cutting token use 27x over competing frameworks. The system treats memory as an interactive graph where agents dynamically refine retrieval paths based on intermediate evidence, using a three-layer Cue-Tag-Content structure and automated ingestion pipelines.
-
Context Windows Are Not Memory: What AI Agent Developers Need to Understand MachineLearningMastery
The article 'Context Windows Are Not Memory' clarifies that a large context window in AI models is not equivalent to memory. It explains how techniques like retrieval, compression, and summarization manage what enters the context window, and how agents can achieve genuine memory persistence.