AI RESEARCH
AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
arXiv CS.LG
•
ArXi:2605.20643v1 Announce Type: new Self-distillation enables language models to learn on-policy from their own trajectories by using the same model as both student and teacher, with the teacher being conditioned on privileged information unavailable to the student. Such information can come in different types or views, such as solutions, nstrations, feedback, or final answers.