Reward-free Alignment for Conflicting Objectives

ArXi:2602.02495v3 Announce Type: replace-cross Direct alignment methods are increasingly used to align large language models (LLMs) with human preferences. However, many real-world alignment problems involve multiple conflicting objectives, where naive aggregation of preferences can lead to unstable