Search papers, labs, and topics across Lattice.
This paper introduces Subspace Tuning (SubT), a novel framework designed to enhance few-shot generalization in Audio-Language Models (ALMs) by addressing the detrimental zero-shot drift in text embedding spaces. By implementing Structured Subspace Parameterization and Residual Anchoring, SubT effectively constrains the adaptation process, preserving the inter-class structure and stabilizing embeddings around their pretrained positions. The method demonstrates significant improvements across 11 audio benchmarks, achieving strong generalization for unseen classes while maintaining computational efficiency by avoiding backpropagation through the text encoder.
Few-shot tuning can distort embeddings, but Subspace Tuning stabilizes adaptation, leading to robust generalization even for unseen classes.
Few-shot adaptation of pretrained Audio--Language Models (ALMs) often improves seen-class performance at the cost of unseen-class generalization, leading to the base-to-new trade-off. We attribute this failure to zero-shot drift in the text embedding space: few-shot tuning can distort inter-class structure and move adapted embeddings far from their pretrained anchors. We therefore propose Subspace Tuning (SubT), a geometry-constrained adaptation framework with two complementary controls on drift. Structured Subspace Parameterization limits structural deformation, and Residual Anchoring stabilizes adaptation around the zero-shot prior. At inference time, Subspace-aware Gating further suppresses negative transfer for weakly aligned unseen classes. Across 11 audio benchmarks, SubT delivers strong few-shot generalization while remaining efficient, operating directly on precomputed text embeddings without text-encoder backpropagation.
