Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolution
Skills, reusable procedural guidance added at inference, can substantially improve LLM downstream performance (Li et al., 2026).
Key points
- Prior work retrieves skills from a bank by semantic relevance, then uses them as inference-time patches or for model distillation.
- We then propose SGUID, a method for selecting a compact subset of skills for distillation.
- Importantly, SGUID supports stable model-skill co-evolution: after a distillation round, a new candidate bank is curated from the updated model's rollouts, and SGUID selects which skills to internalize next.
- These results identify skill selection as the key mechanism for stable model-skill co-evolution.
Sources (1)
- [1]Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-EvolutionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:27 PM
Skills, reusable procedural guidance added at inference, can substantially improve LLM downstream performance (Li et al., 2026).
Prior work retrieves skills from a bank by semantic relevance, then uses them as inference-time patches or for model distillation.
Extractive summary: sentences quoted from the sources.