AI RESEARCH
Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory
arXiv CS.AI
•
ArXi:2602.00521v2 Announce Type: replace While LLM-as-a-Judge is widely used in automated evaluation, existing validation practices primarily operate at the level of observed outputs, offering limited insight into whether LLM judges themselves function as stable and reliable measurement instruments. To address this limitation, we