Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
AutoAdapt addresses the high cost of extending instruction-tuned models to new domains by introducing a modular framework that trains independent Low-Rank Adaptation (LoRA) adapters for each discovered domain. This approach enables targeted, single-adapter training without modifying existing adapters or requiring full model retraining, achieving parity with monolithic LoRA training across diverse benchmarks. The parameter-free routing mechanism and inherent prevention of domain interference by construction facilitate taxonomy-free domain specialization with no aggregate performance loss.
Instruction-tuned models can achieve monolithic training performance on new domains via parallel, independent LoRA adapter training and parameter-free routing, eliminating costly full-model retraining.
Instruction-tuned models are deployed into environments where domains are heterogeneous and evolve, yet adding new domains or data typically requires costly retraining. We present AutoAdapt, a modular framework that incorporates new domains and data via targeted single-adapter training without modifying other adapters. The framework automatically discovers latent domains, uses them to train per-domain Low-Rank Adaptation (LoRA) adapters independently in parallel and performs parameter-free routing. Across 14 domain-specific benchmarks and GPT-4o pairwise judgements, AutoAdapt achieves parity with a LoRA adapter trained on all domains without requiring full-model retraining. We also find evidence of specialisation effect convergence across independent discovery methods. Overall, training each adapter on its own domain prevents domain interference by construction, thus enabling modular, taxonomy-free domain specialisation without aggregate performance loss or full model retraining.