InfoSynth: Information-Guided Benchmark Synthesis for LLMs

ArXi:2601.00575v2 Announce Type: replace Large language models (LLMs) have nstrated significant advancements in reasoning and code generation, but efficiently creating new benchmarks to evaluate these capabilities remains a challenge. Traditional benchmark creation relies on manual human effort, which is expensive and time-consuming. Furthermore, existing benchmarks often contaminate