AI RESEARCH
MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation
arXiv CS.CL
•
ArXi:2605.27186v1 Announce Type: new Large language models often solve tasks from a fully specified prompt but degrade when the same requirements unfold over multiple turns, known as the lost-in-conversation (LiC) gap. We trace part of this degradation to self-contamination: intermediate assistant replies enter later context and carry early deviations forward. Motivated by this mechanism, we propose MAIGO, an on-policy self-distillation method that reduces this contamination using history-cleaned references from the model's own policy.