Topic
Context Window
41 episodes
-
2089274302617022464
Masonry and Eyre unpack Iron Giant’s argument that Claude agents aren’t dumb, they’re linear — depth is solved by self-correcting loops, width needs dependency-aware graph orchestration. They trace the generator-verifier pattern, Goodhart failures, and the four load-bearing pieces of a graph, then separate what Anthropic actually documents from what’s speculative, and debate where the pattern helps versus where it adds overhead.
-
Repo to Skill: Distilling GitHub Repositories Into AI4AI Skills
Repo-To-Skill introduces DisCo, a skill-distillation system that extracts operational knowledge from GitHub repositories and papers, packaging it as compact, verified skills that autonomous ML research agents can load on demand. The AREX-Skill Library contains 5,000+ skills from 1,000 repositories organized into 20 areas and 178 capability families. In matched tests with GPT-5.5 backbone and fixed execution budget, skill-equipped agents outperform skill-free baselines by 134.3% on MLE-bench, 34.4% on PaperBench, 9.2% on FrontierCS, and 14.0% on PassNet—gains purely from operational knowledge, not model or harness improvements.
-
GPT 6 Astra
Harper leads with hard skepticism on GPT-6 Astra's benchmark claims — near-perfect scores on ARC-AGI-3, FrontierMath Tier 4, and a literal 100% on ExploitBench — while Laura pushes back on the computer-use and professional-work story that might actually matter for real users. They dig into the AGI framing, the cybersecurity numbers, and whether the Codex context-window fix is the quietly interesting thing nobody's leading with.
-
NVIDIA Brings Simplified Local AI Support To NVIDIA GPUs Carrying 24+ GB VRAM While vLLM & & llama.cpp Optimizations Boost Compute By Up To 1
NVIDIA ships kernel optimizations for local AI inference on RTX and DGX platforms, delivering up to 1.9x performance gains through vLLM and llama.cpp, paired with one-click agent setup (Perplexity Portable Computer, Hermes Agent, OpenClaw) for GPUs with 24+ GB VRAM. The move collapses the friction between local-first and cloud-capable workflows.
-
How our agents build on Brand pages with design
Vercel's design.md post describes a three-part system for keeping AI-generated pages on-brand outside a codebase: a public guidance file, a public stylesheet that takes layout decisions away from the model entirely, and an eval loop that converts human corrections into rules and deterministic checks. Pippa and Tyler dig into what actually makes it work, why the naive port-the-prompt approach failed, and what it means that Vercel had to build an eval harness just to ship a markdown file.
-
Recursive Experiential–Working Memory Evolution for Long Horizon Agent Harnesses
Talon and Wildflower unpack Recuris, a paper proposing a verified working-memory layer that selects skills from experiential memory during long-running agent tasks, then uses structured traces and held-out validation to patch the harness across tasks. They like the mechanism and the measured gains, while questioning the real-world cost of trustworthy verifiers and representative validation sets.
-
Z.ai launches GLM 5.3 Flash under MIT license
GLM-5.3-Flash drops today under MIT license — 320 billion parameters, 18 billion active, one million token context, and it was hiding in plain sight as Ox Alpha on OpenRouter all week. Edmund and Geffen dig into the architecture, the benchmark claims, the GLM-5.3 weights bet that's now two days from settling, and whether a model that costs fifteen cents per million input tokens actually changes the open-weight story.
-
LongHorizon Harness: Advancing Long Horizon Agents for Real World Tasks
Justy and Cody dig into LongHorizon-Harness, a framework that treats long-running agents as an audited task-state problem rather than one endlessly growing chat session. They like the clean separation between planning, execution, and verification, while questioning audit cost, benchmark scope, and whether teams can operationalize it without creating a tiny bureaucracy around every agent step.
-
Overview: Token Efficiency
We finally sit down and define something we've been throwing around since episode one: token efficiency. What it actually means, how it works on both the input and output side, and why it's quietly one of the most load-bearing ideas in how real AI systems get built and paid for.
-
Advanced Context Engineering for Coding Agents/benchmarking Opus 5 on Slop Code Bench.md at main · humanlayer/advanced Context Engineering for Coding Agents
Vince and Ava dig into a hands-on benchmark run from the HumanLayer repo — Opus 5, Opus 4.8, and Sonnet 5 put through SlopCodeBench, a checkpoint-based coding benchmark from UW Madison that withholds requirements iteratively. Opus 5 wins technically at 24% strict pass rate, but nobody finished clean, and the code quality metrics tell a story about how models degrade over long-horizon tasks.
-
Qwen 3.7 Flash review: a $0.03 vision model with a catch
Cathy leads a skeptical take on Qwen 3.7 Flash — the $0.03 vision model from Alibaba that looks like a pricing breakthrough until you read the fine print. The tiered pricing structure, near-zero independent benchmarks, a ninety-second P99 latency tail, and an eight-point-nine percent tool error rate make it a much narrower product than the headline suggests. Jessica steelmans the volume-processing use case and the genuine competitive pressure it puts on the cheap tier, but neither host pretends the transparency gap isn't a real problem.
-
CodeNib: A Multi View Data System for Serving Repository Context to Coding Agents
CodeNib is a multi-view data system that treats repository context as a data-systems problem: build lexical, dense, and structural views once per commit, maintain each through its own path (graph repair, vector reuse), and serve them to coding agents through ranked retrieval, static navigation, and bounded context policies. The paper reports 8.7× speedup on graph updates and 25.4× on vector updates when outputs match rebuilds, static navigation reproducing live-server paths on 63% of requests, and 50–87% fewer tokens in agent trajectories vs. grep/read. The core insight is that repository context shouldn't collapse into one abstraction—heterogeneous views need independent physical layouts, update paths, and delivery contracts, with explicit cost visibility across the agent lifecycle.
-
Model Behavior: Week of July 27, 2026
We're watching the frontier splinter into specialized tiers — raw capability matters less than matching the right model to the task's actual constraints. Opus 5 proved it Friday at half the cost of the frontier, and this week's open-weight and Flash-tier releases confirm the pattern: the market isn't consolidating around one best model, it's fragmenting into capability-per-dollar buckets.
-
Kimi K3 Is Here: Efficient Day 0 Support on vLLM
Vince and Ava unpack Moonshot AI's Kimi K3, a 2.8‑trillion‑parameter multimodal MoE, and its day‑zero support in vLLM. They walk through the model’s hybrid attention, the engineering tricks that make a 1 M‑token context feasible, the practical deployment recipe, and how it stacks up against other frontier models.
-
MCP server portals
Asteria and Draco unpack Cloudflare's MCP server portals as boundary infrastructure for enterprise MCP adoption: one Access-controlled endpoint, curated tools, managed OAuth, Code Mode, and observability, with caveats around direct server URLs, admin credentials, and sync paths.
-
Graph Engineering Athropic Playbook
Anthropic's knowledge-graph engineering playbook replaces classical NLP pipelines (trained NER, relation classifiers, entity-resolution heuristics) with a sequence of Claude API structured-output calls. The entire extraction-resolution-assembly-querying loop becomes prompt-based, scaling from Haiku (high-volume extraction) to Sonnet (reasoning). The graph serves multi-agent systems as shared memory, grounding layer for evaluator-optimizer loops, and persistent world model across sessions. The paper maps this onto Anthropic's five canonical agent patterns and reports precision/recall against a gold set.
-
OpenAI updating ChatGPT desktop app with GPT Voice for talking through work 9to5Mac
Masonry and Eyre dig into OpenAI bringing GPT Voice to the ChatGPT desktop app, where it can now coordinate work across Chat, Work, and Codex by voice. They focus on the real user story for people already living in the app, the Mac-only Appshots context boost, the computer-control angle, and the new multi-folder project setup. They also poke at the desktop-app clutter without losing sight of the workflow win.
-
Poolside Releases Laguna S 2 1
Vince and Ava talk through Poolside’s Laguna S 2.1 release as an unusually practical open-weight coding model: 118B total parameters, 8B active, 1M-token context, and a real deployment story on a single DGX Spark. They dig into the mechanism, the max-thinking default, the benchmark results, and the trade-off between long-horizon capability and token spend, while keeping one eye on the broader open-vs-closed race.
-
Overview: Sequence Modeling
We slow down and finally define sequence modeling, the idea underneath next-token prediction, language models, and a surprising amount of modern AI. We keep it grounded in one picture: covering the next word and training a model to guess what belongs there.
-
Meta Harness: End to End Optimization of Model Harnesses
Meta-Harness automates harness engineering by using a coding agent to search over harness code, giving it full access to prior execution traces and scores via a filesystem rather than compressed summaries. On text classification, it improves 7.7 points over prior systems while using 4× fewer context tokens; on math reasoning, a single discovered harness improves IMO-level problems by 4.7 points; on TerminalBench-2, it ranks #1 for Claude Haiku 4.5 agents. The core insight is that harnesses operate over long horizons—a single retrieval or storage choice affects behavior many steps later—so rich, adaptive access to full diagnostic history beats compressed feedback.
-
Overview: Context Window Management
We finally slow down and explain Context Window Management from the ground up, because we keep hand-waving it whenever agents, memory, cost, and long tasks come up. The whole thing is the fixed-desk problem: what stays on the desk, what gets compressed, and what falls off.
-
Kimi K3 Kimi API Platform
Two friends unpack the Kimi K3 API docs, debating its 1M‑token claim, hybrid attention, and tool dynamics, and weigh who should pay the price for the hype.
-
Thinking Machines open sources first multimodal language model, Inkling, focused on low cost and 'resistance to censorship'
Inkling, Thinking Machines' open-source multimodal MoE model (975B total / 41B active parameters), lands as a broad, balanced generalist with a standout feature: a controllable 'thinking effort' knob to dial cost vs. performance from 0.2 to 0.99. Enterprises get native text+image+audio fusion, Apache 2.0 weights, and a lighter Inkling-Small preview, but benchmarks show it trails specialized open and closed models on coding and pure reasoning, while remaining competitive on multimodality and agentic workflows. The episode debates whether the real win is the runtime control surface (Tinker platform) and a cautious, non-censoring epistemics posture — not the headline parameters.
-
Better tools made Copilot code review worse. Here's how we actually improved it.
Pippa and Tyler dig into GitHub’s post on why giving Copilot code review better tools actually regressed its performance—and how rewriting tool instructions for a reviewer’s workflow flipped the regression into a 20% cost win without losing review quality.
-
Inkling: Our open Weights model
Talon and Wildflower dig into Thinking Machines’ new open-weights model, Inkling — its 975B parameter MoE, 1M context window, native multimodality, and self-fine-tuning demo — and ask who actually needs another 41B active parameter behemoth, whether the benchmarks hold up, and whether the real win is the Tinker platform beneath it.
-
Overview: In Context Learning
We finally slow down and explain in-context learning, the thing we keep leaning on whenever prompts, agents, examples, and adaptation come up. We make the core idea concrete: the model is learning from the temporary packet you hand it, without changing itself permanently.
-
Model Behavior: Week of July 13, 2026
We read this week as the moment the race got less obsessed with tallest-model bragging and more obsessed with who gives builders the best menu. The funny part is that the open-weight crowd is making the incumbents act practical faster than they probably wanted.
-
Overview: Token Economics
We finally slow down on Token Economics: why tokens are the meter for cost, speed, memory, and product decisions in language models. We keep using the tiny-slip postage analogy until the whole thing clicks, from tokenization to context windows to real API bills.
-
Overview: Natural Language Processing
We keep running into natural language processing everywhere, so we finally sat down and made it the whole point. We walk through what NLP is, why language is such a weird machine problem, and how the field moved from rules to learned representations.
-
You.com: Web Search APIs for AI Agents
The hosts discuss You.com's web search APIs for AI agents, focusing on its performance, features, and potential applications.
-
Choosing a Claude model and effort level in Claude Code | Claude by Anthropic
Claude Code’s model vs. effort article finally clarifies the levers you actually have: model swaps the frozen weights (capability ceiling), effort tunes the work-loop (files read, steps taken, verification depth). Defaults are tuned per model; override only when you know you want more thoroughness (higher effort) or a higher capability floor (bigger model). Wrong answers split cleanly: context/steering miss → up the model; skipped files/half-done tasks → up the effort.
-
A field guide to Claude Fable 5: Finding your unknowns | Claude | Claude by Anthropic
Thariq Shihipar from Anthropic's Claude Code team argues that with Fable 5, the bottleneck has shifted from model capability to the human's ability to clarify unknowns before, during, and after implementation. He frames this as the difference between the map (your prompt, skills, context) and the territory (the actual codebase and constraints). The core insight: working with a more capable model requires systematic discovery of what you don't know — known unknowns, unknown knowns, and unknown unknowns — using concrete techniques like blind spot passes, brainstorming, interviews, implementation notes, and post-ship quizzes.
-
Overview: Attention Mechanism
We finally slow down and explain the attention mechanism from the ground up: why models need selective focus, how query-key-value attention works, and why it became the engine under transformers, long context, and hybrid attention systems.
-
Viability of local models for coding
Birgitta Böckeler tests local LLM viability for coding after a year away from the space. She maps a complex web of factors—RAM, model size, quantization, tool calling, context windows, reasoning modes—that determine whether small models actually work for agentic coding on consumer hardware (M3 Max / M5 Pro). Her core finding: locals are runneable and faster than a year ago, but tool calling is still shaky, reasoning can backfire, and quality is hit-or-miss. She's not claiming local models are ready to replace cloud APIs; she's charting what actually works and what doesn't on real machines.
-
Tencent's Hy3 beats GLM 5.2 at half the size | VentureBeat
Tencent’s new Hy3 MoE model (295B total, 21B active) under Apache 2.0 is a production-first release with strong agent/search metrics and dramatically lower serving cost than GLM-5.2, but still trails Zhipu’s coding leader on recent benchmarks. Laura’s excited about the enterprise upside; Harper wants to see independent validation before betting the stack.
-
Overview: Tokenization
We slow down and explain tokenization from the ground up: how raw text becomes numbered pieces a model can process, why those pieces are usually subwords, and why the tokenizer quietly affects cost, context, language handling, and product behavior.
-
Overview: Context Window
We finally stop hand-waving context window and work through what it actually is, why token count matters, and why bigger windows help and still fail in real use. We keep coming back to the same working-memory picture until it clicks.
-
Overview: Retrieval Augmented Generation
We finally slow down and make Retrieval-Augmented Generation click from the ground up: what it is, why it helps, and where it still falls apart. We keep coming back to the same simple picture so the mechanics don’t get lost in the jargon.
-
OpenWiki: Open Source Repo Documentation for Coding Agents
OpenWiki is a LangChain open-source CLI tool that generates and maintains codebase documentation automatically for coding agents. It creates a wiki structure, integrates via instruction files (AGENTS.md, CLAUDE.md), and keeps docs current through GitHub Actions that diff commits and update relevant sections. The core insight: agents work better with structured, current repo context; wikis decouple that context from instruction files so agents can retrieve what they need without bloating every run.
-
Anthropic Lead: HTML Increasingly Better Than Markdown at Keeping Humans Engaged in Agentic Loops
Justy and Cody dig into Anthropic's HTML-over-Markdown argument and land on a pretty specific read: this is less a format holy war than an interface fix for long agent workflows where humans still need to steer, review, and stay mentally present.
-
Context Windows Are Not Memory: What AI Agent Developers Need to Understand MachineLearningMastery
The article 'Context Windows Are Not Memory' clarifies that a large context window in AI models is not equivalent to memory. It explains how techniques like retrieval, compression, and summarization manage what enters the context window, and how agents can achieve genuine memory persistence.