AI RESEARCH
Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
arXiv CS.LG
•
ArXi:2605.26484v1 Announce Type: new Model merging has emerged as a lightweight paradigm for enhancing Large Language Models (LLMs), yet its underlying mechanisms remain poorly understood. In this work, we analyze late-stage pre-