AI RESEARCH
RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulation
arXiv CS.AI
•
ArXi:2605.20204v1 Announce Type: cross LLM-based user simulation is the primary mechanism for end-to-end agent evaluation, yet simulated users are poor proxies for real humans: unconstrained LLM defaults produce a Formalism Ceiling (style match rates of 6-8% against real users), while hand-crafted behavioral directives trigger Directive Amplification, where models hyper-interpret instructions into unnatural behavioral extremes that vary dramatically across simulator models. We present RealUserSim, the first user simulation framework grounded in real behavioral data.