AI RESEARCH
Review Arcade: On the Human Alignment and Gameability of LLM Reviews
arXiv CS.AI
•
ArXi:2605.28897v1 Announce Type: new LLM-generated reviews for scientific papers are gaining considerable traction and are even being officially piloted by major conferences. We have to assume that not only reviewers are using LLM-assistance, but also that authors use LLMs to revise their papers before submitting. In this work, we perform empirical experiments on papers from the 2025 ACL Rolling Review (ARR) to evaluate LLM reviews from both the author and the reviewer perspective. First, we identify a limited alignment of LLM reviews with human ones.