AI RESEARCH
Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs
arXiv CS.CL
•
ArXi:2605.23975v1 Announce Type: new Audio large language models (Audio LLMs) exhibit systematic failures in transcribing code-switching speech despite strong multilingual capabilities. Focusing on English-Mandarin, we identify three failure modes: language omission, translation-instead-of-transcription, and hallucination. We apply Direct Preference Optimization (DPO) to align models, constructing preference pairs in which chosen responses preserve mixed-language content while rejected responses mimic failure patterns.