AI RESEARCH
What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025
arXiv CS.CL
•
ArXi:2601.07648v2 Announce Type: replace As Natural Language Generation (NLG) dominates modern NLP, scalable evaluation remains a critical bottleneck. Consequently, LLM-as-a-judge (LaaJ) adoption has accelerated rapidly, appearing in papers than human evaluation in 2025. This pivotal shift motivates a critical analysis of current evaluation practices. Overcoming the limits of rigid keyword filtering and manual review, we employ a multi-LLM information extraction pipeline to gather structured metadata from 14,171 papers across four major NLP conferences (2020-2025.