AI RESEARCH
Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
arXiv CS.CL
•
ArXi:2509.10515v1 Announce Type: cross Offline preference optimization methods are efficient for large language models (LLMs) alignment. Direct Preference optimization (DPO)-like learning, one of the most popular approaches, stands out for its efficiency in reward modeling. However, these methods typically follow the convention to use Bradley-Terry (BT) reward modeling that faces several critical assumptions, including the requirement for pairwise