AI RESEARCH
Semiparametric Preference Optimization: Your Language Model is Secretly a Single-Index Model
arXiv CS.AI
•
ArXi:2512.21917v3 Announce Type: replace-cross Policy alignment to preference data typically assumes a known link function between observed preferences and latent rewards (e.g., Bradley-Terry model / logistic link). Misspecification of this link can bias inferred rewards and misalign learned policies. We study policy alignment under an unknown and unrestricted link function.