AI RESEARCH
Consistently Informative Soft-Label Temperature for Knowledge Distillation
arXiv CS.LG
•
ArXi:2605.20357v1 Announce Type: new Knowledge distillation (KD) transfers knowledge from a high-capacity teacher to a compact student by matching their predictive distributions, with temperature scaling serving as a central mechanism for smoothing teacher predictions and exposing informative "dark knowledge" beyond the hard label. However, the standard fixed-temperature design is inherently sample-agnostic.