Direct Preference Optimization

Overview: Group Relative Policy Optimization (GRPO)

We finally slow down on Group Relative Policy Optimization, or GRPO, because it keeps showing up underneath reasoning-model training, chain-of-thought work, and agent tuning. We build it from the mini-tournament intuition through the actual update rule, then place it in where model training seems to be right now.