AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility

Key points

• Instruction-tuned models are deployed into environments where domains are heterogeneous and evolve, yet adding new domains or data typically requires costly retraining.1
• The framework automatically discovers latent domains, uses them to train per-domain Low-Rank Adaptation (LoRA) adapters independently in parallel and performs parameter-free routing.1
• Across 14 domain-specific benchmarks and GPT-4o pairwise judgements, AutoAdapt achieves parity with a LoRA adapter trained on all domains without requiring full-model retraining.1
• We also find evidence of specialisation effect convergence across independent discovery methods.1

Sources (1)

• [1]AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 04:26 PM

  We present AutoAdapt, a modular framework that incorporates new domains and data via targeted single-adapter training without modifying other adapters.

  Instruction-tuned models are deployed into environments where domains are heterogeneous and evolve, yet adding new domains or data typically requires costly retraining.

Extractive summary: sentences quoted from the sources.