AI RESEARCH
Where Rollouts Begin: Low-Load, High-Leverage First-Token Diversification for RLVR
arXiv CS.AI
•
ArXi:2605.28295v1 Announce Type: new Reinforcement Learning with Verifiable Rewards (RLVR) trains reasoning models without labeled trajectories, relying on grouped rollouts to expose the policy to alternative reasoning paths and a verifier to score them. Rollout diversity has accordingly emerged as a central bottleneck in RLVR, with most existing methods broadening exploration through temperature, prefix, or rollout-selection adjustments. We identify a structurally distinguished but overlooked position for broadening this diversity: the first token after the reasoning marker.