Overcoming Prior Barriers: Supervised Fine-Tuning under Long-Tail Distribution

Key points

• We introduce a novel notion named prior barrier to quantify how strongly the pretrained model supports competing concepts over the target concept.1
• We observe that prior barriers follow a long-tail distribution, placing head and tail concepts at different starting points for SFT: head concepts face lower prior barriers, whereas tail concepts require additional instructions to overcome their higher prior barriers.1
• Our theoretical analysis further derives a predictive risk bound for SFT under long-tail prior barriers, explicitly characterizing how the prior barrier and accumulated SFT evidence jointly determine predictive performance.1
• Motivated by this prior barrier-dependent demand, we propose PASS, an adaptive SFT instruction selection method that constructs reference-derived concepts and estimates the distinguishing evidence provided by each instruction, and adaptively allocates the selection budget toward concepts that remain insufficiently covered under the current selection.1

Sources (1)

• [1]Overcoming Prior Barriers: Supervised Fine-Tuning under Long-Tail Distribution
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:16 PM

  Supervised fine-tuning (SFT) adapts pretrained large language models (LLMs) to downstream tasks, but the required concepts can receive substantially different levels of pretrained support.

  We introduce a novel notion named prior barrier to quantify how strongly the pretrained model supports competing concepts over the target concept.

Extractive summary: sentences quoted from the sources.