Search papers, labs, and topics across Lattice.
The paper introduces MUKA, a multi-kernel adaptation framework for few-shot adaptation of Large Audio-Language Models (ALMs). MUKA combines fine-grained representations from instruction-tuned models with global semantic representations from contrastive pretraining using a product kernel. Experiments across 11 audio datasets show MUKA achieves state-of-the-art performance among training-free methods and surpasses training-based adapters in some cases.
Skip the training: MUKA, a multi-kernel adaptation framework, lets Large Audio-Language Models achieve state-of-the-art few-shot performance without any additional training.
Multimodal foundation models have demonstrated impressive generalization capabilities, yet efficiently adapting them to new tasks in a few-shot setting remains a critical challenge. In this work, we investigate the few-shot adaptation of Large Audio-Language Models (ALMs) through both training-based and training-free approaches. We introduce MUKA, a multi-kernel adaptation framework that combines the fine-grained, context-dependent representations of instruction-tuning based models like Pengi with the global semantic representations of contrastive pretraining methods like CLAP. By constructing a product kernel that aligns local similarity with global semantics, MUKA enhances representational power while preserving the theoretical guarantees of kernel methods and avoiding additional training. Extensive experiments across 11 diverse audio datasets demonstrate that MUKA achieves state-of-the-art performance among training-free methods and even surpasses training-based adapters in several scenarios, offering a compelling balance between adaptability and efficiency.
