Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature

ArXi:2602.17385v3 Announce Type: replace Task Arithmetic yields a modular, scalable way to adapt foundation models. Combining multiple task vectors, however, can lead to cross-task interference, causing representation drift and degraded performance. Representation drift regularization provides a natural remedy to disentangle task vectors; however, existing approaches typically require external task data, conflicting with modularity and data availability constraints (e.g., privacy requirements.