Exploring Next / Topics / Kl Divergence Topic Kl Divergence 1 episode Ep 1164 Nov 13, 2026 Theoretical Limits of Language Model Alignment The paper derives a theoretical ceiling for alignment under a KL-divergence budget, offers a way to estimate that ceiling before training, and finds that best-of-N sampling gets close while PPO and GRPO leave substantial performance on the table. Justy and Cody explore what this means for reward hacking, production alignment, and the cost of inference-time search. AI SafetyTrainingReinforcement Learning From Human FeedbackReward Hacking