AI RESEARCH
Speculative Decoding Across Languages
arXiv CS.LG
•
ArXi:2605.30580v1 Announce Type: cross Speculative decoding has become a crucial component of large language model (LLM) inference, enabling faster generation by drafting multiple tokens and verifying them in parallel. However, small draft models tend to suffer from disproportionately poor multilingual capabilities. Thus, when generating text in a non-English language, speculative decoding is far less effective.