AI RESEARCH
Distributed Direct Preference Optimization
arXiv CS.LG
•
ArXi:2605.20696v1 Announce Type: new Preference-based reinforcement learning (RL) is a key paradigm for aligning policies with human judgments, yet its theoretical behavior in distributed settings where preference data are fragmented across heterogeneous users remains poorly understood. Direct Preference Optimization (DPO) avoids explicit reward modeling but lacks convergence guarantees under federated and decentralized