Continual Learning without Continual Training
Instead, we propose replacing continual training with continual inference: a PFN-based model that is meta-trained, and then frozen, adapting to new classes only by extending an in-context evidence set.
ProofPaper ↗
Key points
- Continual learning requires models to adapt to new domains and new classes while retaining prior knowledge.
- Our model, Latent Concept PFN, performs in-context Bayesian inference over a latent concept space that captures semantic structure shared across domains and classes.
- The same method handles both domain and class incremental continual learning without task identity.
- Experiments on class and domain incremental learning datasets demonstrate competitive continual learning performance while learning interpretable latent concepts.
Sources (1)
- [1]Continual Learning without Continual TrainingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 04:39 PM
Instead, we propose replacing continual training with continual inference: a PFN-based model that is meta-trained, and then frozen, adapting to new classes only by extending an in-context evidence set.
Continual learning requires models to adapt to new domains and new classes while retaining prior knowledge.
Extractive summary: sentences quoted from the sources.