AI RESEARCH
From One-Pass SGD to Data Reuse: Mini-Batch Scaling Laws in Sketched Linear Regression
arXiv CS.LG
•
ArXi:2605.24316v1 Announce Type: new Scaling laws provide compact descriptions of how prediction error varies with compute, model size, and data, but existing theory mainly treats single-sample SGD or full data reuse, leaving the role of mini-batching unclear. We study batch scaling laws for sketched linear regression under a power-law covariance spectrum and a source condition on the target parameter. We analyze one-pass batch SGD, multi-pass batch SGD with replacement, and multi-pass batch SGD without replacement.