Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling

ArXi:2602.10623v2 Announce Type: replace-cross Reward models learned from human preferences are central to aligning large language models (LLMs) via reinforcement learning from human feedback, yet they are often vulnerable to reward hacking due to noisy annotations and systematic biases such as response length or style. We propose Bayesian Non-Negative Reward Model (BNRM), a principled reward modeling framework that integrates non-negative factor analysis into Bradley-Terry (BT) preference model.