Search papers, labs, and topics across Lattice.
This paper introduces Self-Routed Tensor Adapters (SRTA), a novel framework for parameter-efficient adaptation of visual foundation models across diverse domains without the need for external routing mechanisms. By utilizing a learnable domain matrix to compute routing weights and blending slices from a shared Tucker core, SRTA achieves domain-aware specialization while maintaining a compact parameter footprint. Experimental results demonstrate that SRTA outperforms or matches the accuracy of existing MoE-style parameter-efficient fine-tuning methods while significantly reducing the number of trainable parameters required.
Achieving competitive accuracy with fewer parameters, Self-Routed Tensor Adapters redefine the landscape of visual model adaptation across heterogeneous domains.
Universal visual representations require adaptation mechanisms that adapt across heterogeneous domains without fragmenting knowledge into domain-specific modules. Parameter-efficient fine-tuning adapts frozen visual foundation models efficiently, but standard low-rank adapters use a fixed subspace for all inputs, which can be restrictive when domains differ in style, background, and semantic context. MoE-based adapters improve specialization through multiple expert pathways, but often rely on external routers and large expert banks, adding parameters and separating routing from adaptation. We propose \textbf{Self-Routed Tensor Adapters}, a compact framework for multi-domain visual adaptation. SRTA projects each input into a low-rank space, computes routing weights from this representation using a learnable domain matrix, and uses these weights to blend slices of a shared Tucker core. This produces a sample-specific adaptation matrix without an external gating network, allowing shared visual factors to be reused while supporting domain-aware specialization. To strengthen pathway learning, we introduce a progressive depth-weighted routing objective that supervises routing decisions across adapter layers. Across five heterogeneous multi-domain visual classification benchmarks, SRTA achieves competitive or slightly stronger average accuracy than MoE-style PEFT baselines while using substantially fewer trainable parameters. At rank 64, SRTA uses 2.77M parameters in the 4-domain setting compared with 9.52M for MoLoRA, and 3.00M in the 6-domain setting compared with 14.31M. Overall, SRTA offers an effective accuracy-parameter trade-off for adapting visual foundation models toward universal multi-domain representations. \href{https://github.com/surajyadav-research/SRTA}{GitHub}
