Search papers, labs, and topics across Lattice.
This study investigates the effectiveness of reasoning-oriented post-training using Low-Rank Adaptation (LoRA) and inference-time LoRA rescaling in Large Audio-Language Models (LALMs) for Audio-Dependent Question Answering (ADQA). By employing a structured Chain-of-Thought framework, the authors decompose the reasoning process and assess how LoRA adaptation impacts two different model architectures, revealing significant variations in performance. The findings indicate that while post-training enhances the Qwen-based systems, it negatively affects MOSS-Audio, with moderate LoRA rescaling improving the Qwen model's accuracy to 61.05% and restoring some performance in MOSS-Audio, which achieved a top-1 accuracy of 67.70%.
Post-training with LoRA can boost accuracy in some models while hindering others, revealing the nuanced interplay between architecture and adaptation in audio-dependent tasks.
Audio-Dependent Question Answering (ADQA) requires Large Audio-Language Models (LALMs) to answer questions whose correct answers depend on the given audio content. Successful ADQA requires accurate audio perception, identification of question-relevant evidence, and cross-modal reasoning. Using the official ADQA dataset of DCASE 2026 Task 5, we investigate reasoning-oriented post-training with Low-Rank Adaptation (LoRA) and inference-time LoRA rescaling for both Qwen2.5-Omni and MOSS-Audio-8B-Thinking. We introduce a structured Chain-of-Thought (CoT) framework that decomposes the reasoning process into question analysis, question type, audio evidence, and reasoning. We then analyze how task-specific LoRA adaptation affects the two backbones and further explore inference-time rescaling of trained LoRA adapters. Experiments on the development set reveal markedly backbone-dependent behavior: post-training improves the Qwen-based systems but substantially degrades MOSS-Audio under our supervised fine-tuning configuration. Moderate LoRA rescaling further improves the best Qwen system's top-1 accuracy from 58.93% to 61.05% and partially restores the performance of the fine-tuned MOSS-Audio models, while the best MOSS-Audio system achieves 67.70% top-1 accuracy. Our submitted systems ranked third overall and second among lightweight systems under 10B parameters in the challenge.
