AutoAdapt is a modular framework that automatically discovers latent domains and trains per-domain Low‑Rank Adaptation (LoRA) adapters independently in parallel, without modifying other adapters. It uses parameter‑free routing to integrate new domains, achieving performance comparable to a LoRA adapter trained on all domains while avoiding full‑model retraining. The approach prevents domain interference by construction, enabling taxonomy‑free domain specialization across 14 benchmarks and GPT‑4o evaluations.
Instruction-tuned models are deployed into environments where domains are heterogeneous and evolve, yet adding new domains or data typically requires costly retraining. We present AutoAdapt, a modular...
arXiv:2608. 05161v1 Announce Type: cross Abstract: Instruction-tuned LLMs are deployed into environments where domains evolve, yet extending a fine-tuned model's capabilities without full retraining remains an unsolved practical challenge.
The paper introduces Domain‑Specialized Upcycling (DS‑Upcycling), a method that expands a dense Transformer with domain‑specific experts and a router to improve multi‑domain adaptation. Unlike prior LoRA‑based adapters that either weight all domains equally or activate all experts, DS‑Upcycling trains a domain‑aware router to select the appropriate expert while freezing other parameters. Experiments demonstrate superior average performance across domains and high routing accuracy (90‑98%) with only 256 examples per domain.
arXiv:2407. 21311v2 Announce Type: replace-cross Abstract: Unsupervised domain adaptation (UDA) aims to mitigate domain shift, where the distribution of labeled source data differs from that of unlabeled target data.
arXiv:2607. 17467v1 Announce Type: cross Abstract: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples.
arXiv:2608. 16384v1 Announce Type: cross Abstract: Universal visual representations require adaptation mechanisms that adapt across heterogeneous domains without fragmenting knowledge into domain-specific modules.
HyperRepair is a framework that restores the general capabilities of fine‑tuned language models while keeping their domain specialization, using only inference time. It learns a hypernetwork from repairs on source models, which generates a Repairing Side Net (RSN) that selectively corrects the model based on activation fingerprints. Evaluated on two model families, 12 LoRA adaptation datasets, and 15 general benchmarks, HyperRepair retains 99.5% of fine‑tuned domain performance and 93.4% of the base model’s general performance.
arXiv:2608. 10804v1 Announce Type: cross Abstract: Deep neural networks excel in various tasks but struggle to generalize across evolving data distributions, leading to significant performance degradation under domain shifts.
arXiv:2609.37362v1 Announce Type: new Abstract: Large language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality--eff...
The paper investigates whether the sparsity of Mixture-of-Experts (MoE) models leads to intrinsic semantic organization across modalities and domains. It shows that experts naturally specialize semantically even without explicit modular training. The authors propose ExpertLens, a data‑free method that decodes router weights to identify domain‑specialized experts, enabling selective fine‑tuning that matches or exceeds full fine‑tuning while updating only 21.7–47.0% of parameters and achieving a 4.0× speedup, outperforming LoRA in both performance and efficiency.
FedSAP is a federated learning framework that addresses heterogeneous edge devices by using structured pruning as a budget-constrained tri-state channel allocation. It partitions model channels into a Global pool, pseudo-domain-specific Private pools, and a Dropped state, allowing broadly useful features to be shared while isolating domain-sensitive updates. Experiments on Digits and Office-Caltech datasets show FedSAP achieving higher mean global accuracy than the strongest baseline while supporting up to 80% client pruning ratios.
arXiv:2511. 18468v2 Announce Type: replace Abstract: Continual Test-Time Adaptation (CTTA) is crucial for deploying models in real-world applications with unseen, evolving target domains.