AI RESEARCH
The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study
arXiv CS.LG
•
ArXi:2605.20767v1 Announce Type: cross Large language models (LLMs) show potential as simulators of human behavior, offering a scalable way to study responses to interventions. However, because LLMs are trained largely on observational data, interventions in experiments with LLM-simulated synthetic users can induce unintended shifts in latent user attributes, causing user drift where the implicit simulated population differs across treatment conditions, potentially distorting effect estimates.