Search papers, labs, and topics across Lattice.
This study investigates the temporal reasoning failures of Large Audio Language Models (LALMs) through a newly introduced benchmark comprising 1,657 questions across three foundational tasks. The research reveals that LALMs often rely more on textual cues than audio inputs, and a causal mechanistic analysis shows that redistributing attention across audio tokens significantly enhances performance. By optimizing attention scaling at bottleneck layers, the accuracy of LALMs improves from 55.9% to 59.1%, highlighting a critical avenue for addressing their limitations in temporal understanding.
LALMs can boost their temporal reasoning accuracy by 3.2% simply by better redistributing attention across audio tokens rather than relying on textual cues.
Large Audio Language Models (LALMs) achieve strong performance on a variety of audio understanding tasks but continue to struggle with temporal reasoning, a fundamental capability central to human auditory perception. Understanding the causes of these failures remains challenging as existing benchmarks report performance gaps without probing underlying mechanisms. To address this, we introduce a benchmark with 1,657 questions across three foundational tasks designed specifically for mechanistic analysis. Examining model outputs across varying input settings (behavioral analysis) reveals that models often under-utilize audio when textual cues are available. We also provide the first causal mechanistic analysis of temporal reasoning failures in LALMs. Comparing attention upweighting against scaling, we find that redistributing attention across audio tokens is more effective than increasing audio attention. Targeting task-relevant tokens yields further gains. These findings suggest that modality imbalance alone cannot explain failures. Attention scaling at bottleneck layers improves accuracy from 55.9% to 59.1% without fine-tuning, demonstrating a promising direction for future work.
