Privileged Context as Drift in On-Policy Self-Distillation

Key points

• Existing work varies what privileged context contains and how it is produced while also changing models, data, and training setups, making the effects of privileged context design difficult to isolate.1
• Motivated by efforts in continual learning to reduce catastrophic forgetting, we study how the choice of privileged context affects policy drift.1
• We train Qwen2.5-7B with OPSD across these nine combinations and three datasets, measuring target-task accuracy, prior-task retention, reverse KL from the base policy, and parameter-update geometry.1
• For continual learning, these findings suggest that privileged context should be treated as part of OPSD's stability design because it is associated with how far and in what direction the policy moves.1

Sources (1)

• [1]Privileged Context as Drift in On-Policy Self-Distillation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 06:45 AM

  On-policy self-distillation (OPSD) trains a language model to match a copy of itself conditioned on privileged context.

  Existing work varies what privileged context contains and how it is produced while also changing models, data, and training setups, making the effects of privileged context design difficult to isolate.

Extractive summary: sentences quoted from the sources.