AI RESEARCH
Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Experts
arXiv CS.LG
•
ArXi:2506.21035v5 Announce Type: replace Continual learning (CL) with large pre-trained models aims to incrementally acquire knowledge without catastrophic forgetting. Existing LoRA-based Mixture-of-Experts (MoE) methods expand capacity by adding isolated new experts while freezing old ones, but still suffer from redundancy, interference, routing ambiguity, and consequent forgetting. We investigate the issues stemming from coarse-grained expert granularity.