AI RESEARCH
From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models
arXiv CS.AI
•
ArXi:2606.03660v1 Announce Type: new Large language models are increasingly used as chemistry assistants, yet most chemistry benchmarks still score only final answers. This masks a critical failure mode: a model may output the correct molecule, product, or option while its reasoning violates chemical logic. Existing process-level evaluators are hard to scale because LLM judges and human step-level process annotation are costly, inconsistent, and vulnerable to hallucination. We