AI RESEARCH
S-SPPO: Semantic-Calibrated Self-Play Preference Optimization
arXiv CS.AI
•
ArXi:2606.01561v1 Announce Type: new Aligning Large Language Models (LLMs) with human preferences is often formulated via Direct Preference Optimization (DPO). However, the standard Bradley-Terry instantiation of DPO is limited in modeling common departures from transitivity in human preferences. To address this, recent work has