AI RESEARCH
Connecting Independently Trained Modes via Layer-Wise Connectivity
arXiv CS.LG
•
ArXi:2505.02604v5 Announce Type: replace Empirical studies have shown that continuous low-loss paths can be constructed between independently trained neural network models. This phenomenon, known as mode connectivity, refers to the existence of such paths between distinct modes-i.e., well-trained solutions in parameter space. However, existing empirical methods do not reliably connect independently trained modes and have been evaluated mainly on a narrow set of architectures (e.g., basic CNNs, VGG, and ResNet), leaving their effectiveness on newer models unclear.