AI RESEARCH
Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
arXiv CS.LG
•
ArXi:2605.21125v1 Announce Type: new Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the reasoning capabilities of large language models (LLMs). However, GRPO is prone to advantage collapse, a failure mode where homogeneous rewards within a group (e.g., all correct or all incorrect answers) yield near-zero advantages and vanishing gradients. To address this, we