Topic
Credit Assignment
6 episodes
-
Overview: Credit Assignment
We throw 'credit assignment' around constantly on this show and realized we've never actually stopped to explain it — so this episode, we fix that. It's the foundational question underneath all of machine learning: when a model gets something right, which of the thousand tiny decisions inside it actually deserved the credit?
-
DART SD: Diamond topology Aware Retrieval and Tuning for Self Distillation of Multi Turn Tool Calling Agents
Edmund and Geffen discuss the ByteDance/USTC paper DART-SD, which tackles 'topological collapse' in agent distillation. They discuss how moving from linear trajectory imitation to a diamond-topology graph (ISTG) allows student models to learn recovery from errors without destroying their own valid reasoning paths.
-
Agent Lightning v1.0: Towards Harnessed Agentic RL
Edmund and Geffen dig into Agent Lightning v1.0, a Microsoft research framework that tackles the underspecified engineering problems of harnessed agentic RL — training a model through the same harness it runs in at deployment. They work through the four core challenges (retokenization, advantage calculation, loss normalization, and scheduling), react to a striking SWE-bench result on modest compute, and argue about whether this is shippable infrastructure or a well-documented research artifact.
-
Progressive Agent Skill Generation via Reinforcement Learning
Vince and Ava dig into Skill-Alpha, a paper on generating reusable agent skills with reinforcement learning. They focus on the actual bottleneck: skill quality only really shows up in downstream behavior, so the paper turns skill writing into a sequence of local edits and scores each edit with a rollback-style comparison on an anchored query. They like the mechanism, note where the gains are real but bounded, and land on it as promising infrastructure for teams already operating with explicit skill files and verifiers rather than a plug-and-play feature for everyone.
-
AREX: Towards a Recursively Self Improving Agent for Deep Research
Pippa and Tyler dig into AREX, a recursively self-improving deep research agent from BAAI that alternates between an inner search loop and an outer constraint-verification loop — and discuss whether that architecture is genuinely novel or a smarter repackaging of ideas the field already had.
-
Seed: Self Evolving On Policy Distillation for Agentic Reinforcement Learning
Seed tackles the credit-assignment problem in long-horizon agent reinforcement learning by turning completed trajectories into evolving natural-language hindsight skills, then distilling their effect into dense token-level training signals. Vince sees a potentially shippable training pattern for teams already running agentic RL; Ava likes the on-policy design but wants stronger evidence that self-generated skills do not amplify the model’s own blind spots.