Search papers, labs, and topics across Lattice.
This study investigates the performance of Large Audio Language Models (LALMs) when processing concatenated audio segments, revealing a significant task-dependent robustness gap. While automatic speech recognition (ASR) maintains stability, audio question answering (AQA) experiences substantial degradation, attributed to differing audio information routing mechanisms. Layer-wise attention knockout analysis indicates that ASR relies on direct retrieval from audio tokens, whereas AQA's performance is hindered by a bottleneck in accessing mediated information during generation, suggesting limitations in information utilization for generalization in LALMs.
A striking task-dependent robustness gap reveals that while ASR thrives on direct audio retrieval, AQA falters due to bottlenecks in mediated information access.
Large Audio Language Models (LALMs) have demonstrated strong performance across a wide range of audio understanding tasks. However, they are typically evaluated on single, coherent audio segments, leaving their behavior under less familiar input configurations underexplored. We study this issue through a controlled setting in which two audio segments are concatenated into a single input. Across multiple LALMs, we observe a striking task-dependent robustness gap: automatic speech recognition (ASR) remains comparatively stable, whereas audio question answering (AQA) degrades substantially. To investigate the mechanisms underlying this disparity, we analyze how audio information is routed through LALM decoders using layer-wise attention knockout. The results reveal distinct task-dependent pathways. ASR relies primarily on direct retrieval from audio tokens by answer tokens, whereas AQA depends more strongly on a mediated route in which audio information is first integrated into prompt tokens and subsequently accessed during generation. We further probe prompt-token representations under audio concatenation and find that task-relevant audio attributes remain readily decodable, particularly in middle and later decoder layers, even when AQA performance deteriorates sharply. This dissociation indicates that the failure cannot be explained by complete loss of audio information from the decoder states and is instead consistent with a downstream bottleneck in retrieving or utilizing prompt-mediated information during answer generation. Together, our findings reveal task-dependent audio information routing in LALMs and highlight information utilization as a potential limitation on their generalization.
