AI RESEARCH
Multi-Step Likelihood-Ratio Correction for Reinforcement Learning with Verifiable Rewards
arXiv CS.LG
•
ArXi:2605.20865v1 Announce Type: new Reinforcement learning with verifiable rewards (RLVR) plays a pivotal role in improving the reasoning ability of large language models. However, widely used PPO surrogate objectives are fundamentally local, as they rely on a local approximation of the exact policy gradient objective. While this approximation improves stability by reducing the variance induced by importance sampling, it also