Efficient Benchmarking Is Just Feature Selection and Multiple Regression

ArXi:2605.25773v1 Announce Type: cross Efficient benchmarking techniques aim to lower the computational cost of evaluating LLMs by predicting full benchmark scores using only a subset of a benchmark's questions. By reframing this problem as an instance of multiple regression with feature selection, we find that existing efficient benchmarking methods can be greatly improved by simply using kernel ridge regression at the prediction stage.