AI RESEARCH
AdvantageFlow: Advantage-Weighted Least Squares for RL in Flow Models
arXiv CS.AI
•
We introduce AdvantageFlow, a forward-process reinforcement learning algorithm for rectified flow models. Unlike Flow-GRPO, which optimizes the reverse process, we optimize an advantage-weighted forward-process prediction loss. We stabilize it by rollout policy regularization, which reduces variance and arises from fitting a local reward-improving target distribution.