Alignment Tuning for Large Language Models: A Data-Centric Lens on Alignment Data Pipelines

ArXi:2605.26442v1 Announce Type: cross Much of the alignment tuning literature is organized around optimization objectives, while the construction of alignment data is often treated implicitly. In this survey, we adopt a data centric perspective and reframe alignment tuning as a pipeline design problem. We decompose alignment data construction into three interacting stages, response synthesis, preference evaluation, and preference instantiation, and use this framework to organize existing alignment methods into a unified taxonomy.