Distillation for Incrimination and Distillation for Capabilities

Key points

• However, distilling such a model into a weaker benign student places the teacher in a Distillation Double Bind: if misalignment transfers, the student may conceal it less effectively, revealing evidence about the teacher; if it does not, the student may learn useful capabilities while remaining benign.1
• We introduce two distinct distillation approaches, one targeting each outcome.1
• Distilling AuditBench's secret-keeping models into their underlying instruction-tuned model produces students that are significantly more likely than their teachers to admit their hidden behavior when asked, suggesting that knowledge of the behavior transferred more readily than the propensity to conceal it.1
• Together, these findings demonstrate two ways distillation can be used for AI safety: incriminating misaligned models, and extracting their capabilities without their misalignment.1

Sources (1)

• [1]Distillation for Incrimination and Distillation for Capabilities
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 11:51 PM

  Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative.

  However, distilling such a model into a weaker benign student places the teacher in a Distillation Double Bind: if misalignment transfers, the student may conceal it less effectively, revealing evidence about the teacher; if it does not, the student may learn useful capabilities while remaining benign.

Extractive summary: sentences quoted from the sources.