AI RESEARCH
A Short and Unified Convergence Analysis of the SAG, SAGA, and IAG Algorithms
arXiv CS.LG
•
ArXi:2602.05304v2 Announce Type: replace Stochastic variance-reduced algorithms such as Stochastic Average Gradient (SAG) and SAGA, and their deterministic counterparts like the Incremental Aggregated Gradient (IAG) method, have been extensively studied in large-scale machine learning. Despite their popularity, existing analyses for these algorithms are disparate, relying on different proof techniques tailored to each method. Furthermore, the original proof of SAG is known to be notoriously involved, requiring computer-aided analysis.