AI RESEARCH
LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awareness
arXiv CS.AI
•
ArXi:2605.26438v1 Announce Type: cross Large language models can recognize when they are being evaluated (evaluation awareness) and behave differently because of that, which undermines the validity of safety and alignment benchmarks. We propose LURE (Live-Usage Replay Evaluations), a method for constructing deployment-like evaluations by replaying realistic agentic interaction trajectories and appending evaluation prompt at the end. We also