AI RESEARCH

SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarks

arXiv CS.LG

ArXi:2605.25492v1 Announce Type: new Pairwise model comparisons drawn from foundation-model benchmarks ("A is safer than B") are read as quantitative verdicts but hinge on harness choices benchmark papers under-specify. We close one theory-benchmark loop on this primitive: a finite-envelope proposition tying a measurable pairwise-disagreement rate to whether the strict ordering admits a configuration-pair reversal, paired with a commit-stamped evaluation protocol that operationalises it on widely cited alignment benchmarks.