SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning

Key points

• Supervised fine-tuning (SFT) equips large language models (LLMs) with specialized capabilities, but often comes at the cost of forgetting the general capabilities of their parent models (i.e., the pretrained models before fine-tuning).1
• This approach also extends beyond parent-SFT pairs: responses from a small open-source SFT model can improve a strong closed-source LLM, outperforming either model alone.1
• Furthermore, we use a Bayesian framework to derive theoretical guarantees that bound the error of SFT-as-context relative to the SFT model on fine-tuned capabilities and to the parent model on general capabilities.1
• In addition, we visualize the attention weights and find that the parent model attends more to useful SFT responses and less to irrelevant ones, suggesting that selective attention helps the parent model use the SFT response through in-context learning.1

Sources (1)

• [1]SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 02:57 AM

  We introduce SFT-as-context, a training-free method in which the parent model uses the SFT model's response as context to answer the query.

  Supervised fine-tuning (SFT) equips large language models (LLMs) with specialized capabilities, but often comes at the cost of forgetting the general capabilities of their parent models (i.e., the pretrained models before fine-tuning).

Extractive summary: sentences quoted from the sources.