AI RESEARCH
MATCHA: Matching Text via Contrastive Semantic Alignment
arXiv CS.CL
•
ArXi:2605.27345v1 Announce Type: new Reliable evaluation is essential for understanding large language model (LLM) performance, yet today's go-to metrics, namely token-overlap scores (e.g., ROUGE) and embedding-based measures (e.g., BERTScore), often misjudge semantic similarity of documents. Our study shows that both token-overlap metrics and embedding-based metrics routinely assign nearly identical scores to texts that directly contradict each other, thereby potentially masking fundamental errors. We.