How Many Human Survey Respondents is a Large Language Model Worth? An Uncertainty Quantification Perspective

ArXi:2502.17773v5 Announce Type: replace-cross Large language models (LLMs) are increasingly used to simulate survey responses, but synthetic data can be misaligned with the human population, leading to unreliable inference. We develop a general framework that converts LLM-simulated responses into reliable confidence sets for population parameters of human responses, quantifying the uncertainty induced by the human-LLM misalignment.