TRACER: Persistent Regularization for Robust Multimodal Finetuning

ArXi:2605.29380v1 Announce Type: cross Mainstream strategies for finetuning pretrained multimodal models often degrade out-of-distribution (OOD) robustness, a phenomenon known as catastrophic forgetting. In this paper, we develop a theoretical framework for multimodal contrastive finetuning, yielding closed-form solutions and a geometric decomposition for each strategy. This framework shows that self-distillation is effective than other regularization approaches to retain the knowledge of the pretrained model.