AI RESEARCH
Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
arXiv CS.AI
•
ArXi:2605.24681v1 Announce Type: cross Large Language Models (LLMs) have shown great promise in multilingual machine translation (MT), even with limited bilingual supervision. However, fine-tuning LLMs with parallel corpora presents major challenges, namely parameter interference. To address these issues, we propose Mix-MoE, a mixed Mixture-of-Experts framework designed to train LLMs for multilingual MT. Our framework operates in two distinct stages: (1) post-pre