ResearchResearch paperLarge Language Models · Training & Scaling2 sources · Oct 8, 2026

Self-Preservation in Language Models

Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities.

Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data.
This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly?
To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model.
Continual instruction tuning with Qwen models of varying sizes yields three findings.

[1]Self-Preservation in Language Models
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 06:24 PM
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities.
Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data.
[2]Self-Preservation in Language Models
Hugging Face Daily Papers · Oct 8, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.

Before this