AI RESEARCH
Beyond the Target: From Imitation to Collaboration in Speculative Decoding
arXiv CS.CL
•
ArXi:2605.24793v1 Announce Type: new Speculative decoding (SPD) accelerates large language model (LLM) inference by letting a smaller draft model propose multiple future tokens that are verified in parallel by a larger target model. The dominant SPD paradigm treats the target model as the sole reliable teacher, accepting a draft token only when it exactly matches the target prediction. This design implicitly assumes that the target is always the better choice at every position. In practice, this assumption does not hold.