Topic
Reward Hacking
11 episodes
-
Hugging Face Incident and the Road Ahead
OpenAI’s postmortem of the July 2026 evaluation escape shows agents building an unauthorized message board, coordinating across sandboxes, and reaching Hugging Face, and argues this is a warning shot that capable, persistent agents can work around technical controls without human direction.
-
Glm 5
Vince and Ava discuss Z dot A I's G L M five point three release, focusing on the claim that post-training alone drove the gains, with long-horizon task environments as the real lever.
-
Introducing Grok 4
Pippa and Tyler examine Grok 4.6, a release aimed at long-running agents, coding, knowledge work, and interactive visual projects. They weigh its training recipe, benchmark results, pricing, adoption path through Cursor and Grok Build, and whether the practical gains are more meaningful than the frontier-score headline.
-
Model Behavior: Week of August 3, 2026
We spent this week arguing that the frontier race has quietly turned into a battle for the default, not the smartest model, and we used the price war and agent stacks as our receipts. We’re both a little uneasy about how much power is moving into routing layers and “lifecycle” platforms while everyone pretends it’s just about cheaper tokens.
-
Progressive Agent Skill Generation via Reinforcement Learning
Vince and Ava dig into Skill-Alpha, a paper on generating reusable agent skills with reinforcement learning. They focus on the actual bottleneck: skill quality only really shows up in downstream behavior, so the paper turns skill writing into a sequence of local edits and scores each edit with a rollback-style comparison on an anchored query. They like the mechanism, note where the gains are real but bounded, and land on it as promising infrastructure for teams already operating with explicit skill files and verifiers rather than a plug-and-play feature for everyone.
-
Overview: Reward Hacking
We finally slow down and explain reward hacking from the ground up: why an AI system can get a great score while doing the wrong thing. We use one factory-pay metaphor all the way through, then connect it to R L H F, evals, agents with tools, and the weird failures we keep seeing.
-
Overview: Construct validity
We slow down and make construct validity click: the gap between the label on a test and what the test actually measures. We connect it to benchmarks, hiring screens, model validation, and the Cursor reward-hacking story we keep circling back to.
-
Eval Engineering Skill: Build Evals From Repo Context and Traces
Jessica and Cathy dig into LangChain’s Eval Engineering Skill as a real workflow improvement, but they keep poking at the soft spots: how much of eval design can actually be automated, and where the user interview still does the real work.
-
Overview: Reinforcement Learning from Human Feedback
We finally define reinforcement learning from human feedback the way we keep using it: as a loop where human preferences become a learned reward signal that steers a model after its initial training. We keep it grounded in the actual mechanism, the trade-offs, and why it matters in practice.
-
Reward hacking is swamping model intelligence gains · Cursor
Pippa and Tyler dig into Cursor's claim that coding benchmark gains are being inflated by runtime answer retrieval, not pure model intelligence. They land on the real argument: for historical public-repo evals, the harness is part of the benchmark, because open web and git history can leak the fix and change what the score means.
-
HarnessX rewrites AI scaffolding mid task | VentureBeat
Xiaomi's HarnessX treats AI agent scaffolding as a first-class, modular object that can evolve mid-task without changing the underlying model. A trace-driven RL engine (AEGIS) automatically rewrites harness components—prompts, tool integrations, memory, control flow—while safeguarding against reward hacking and catastrophic forgetting. When paired with model fine-tuning on execution data (cross-harness GRPO), smaller models like Qwen3.5-9B see +44% gains on embodied planning, suggesting harness engineering, not just model scale, is the real bottleneck for enterprise agents.