Topic
Reinforcement Learning From Human Feedback
12 episodes
-
AREX: Towards a Recursively Self Improving Agent for Deep Research
Pippa and Tyler dig into AREX, a recursively self-improving deep research agent from BAAI that alternates between an inner search loop and an outer constraint-verification loop — and discuss whether that architecture is genuinely novel or a smarter repackaging of ideas the field already had.
-
GitHub FareedKhan Dev/train LLM From scratch: A straightforward method for training your LLM, from downloading data to generating text.
A single-GPU end-to-end LLM training guide lands on GitHub—hand-written PyTorch, pretrain to chat in one repo, plus full RLHF. Vince is giddy; Ava wants to know which corner of the GPU shelf this actually runs on. Build Next shows the exact CLI to kick it off on a T4.
-
Poolside Releases Laguna S 2 1
Vince and Ava talk through Poolside’s Laguna S 2.1 release as an unusually practical open-weight coding model: 118B total parameters, 8B active, 1M-token context, and a real deployment story on a single DGX Spark. They dig into the mechanism, the max-thinking default, the benchmark results, and the trade-off between long-horizon capability and token spend, while keeping one eye on the broader open-vs-closed race.
-
Overview: Supervised Fine Tuning
We finally slow down and make supervised fine-tuning click, because we keep leaning on S F T like everyone already has the whole shape of it. We build it from the apprentice-and-worked-examples picture into the actual training loop, the examples, and the trade-offs.
-
Overview: Retry Loops and Error Recovery
We finally define retry loops and error recovery, because we keep tossing the term around like everybody knows exactly what it means. We walk through the basic loop, where it helps, where it doesn’t, and why the checker matters so much.
-
Alibabas Tongyi Lab Releases Qwen Audio 3 0 TTS a Hosted Text to Speech Model in Flash and Plus Tiers Across 16 Languages
Cooper and Miles examine Alibaba's Qwen-Audio-3.0-TTS, comparing its Flash and Plus tiers, multilingual support, voice controls, architecture, hosted-only trade-offs, pricing, and real production use cases.
-
Seed: Self Evolving On Policy Distillation for Agentic Reinforcement Learning
Seed tackles the credit-assignment problem in long-horizon agent reinforcement learning by turning completed trajectories into evolving natural-language hindsight skills, then distilling their effect into dense token-level training signals. Vince sees a potentially shippable training pattern for teams already running agentic RL; Ava likes the on-policy design but wants stronger evidence that self-generated skills do not amplify the model’s own blind spots.
-
Stanford Researchers Introduce TRACE: A Capability Targeted Agentic Training System That Turns Recurrent Agent Failures Into Synthetic RL Environment
Stanford's TRACE system converts recurring agent failures into targeted synthetic training environments, using LoRA experts and MoE routing to close specific capability gaps without retraining the whole model.
-
Overview: Reinforcement Learning from Human Feedback
We finally define reinforcement learning from human feedback the way we keep using it: as a loop where human preferences become a learned reward signal that steers a model after its initial training. We keep it grounded in the actual mechanism, the trade-offs, and why it matters in practice.
-
Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs
Jessica and Cathy dig into RL with Metacognitive Feedback (RLMF): a post-training loop that rewards models not just for correct answers, but for accurately judging how well they did—improving both task performance and the faithfulness of uncertainty expressions. They explain the mechanism (metacognitive data selection and metacognitive advantage scaling), discuss trade-offs, and debate whether this is still research-only or actually shippable.
-
What Is the Ultra Code Mode in Claude Code? X High Effort Plus Dynamic Workflows
Justy and Cody discuss Ultra Code mode in Claude Code, treating it as a real product-shaped escalation from solo coding assistant to higher-effort, multi-agent coding workflow, while staying skeptical about claims around automatic coordination.
-
Introducing OpenRL: A self Hosted post training API for fine tuning LLMs | Google Open Source Blog
Justy and Cody discuss Google’s OpenRL, a self-hosted post-training API that tries to separate RL research loops from the Kubernetes and GPU infrastructure underneath them.