Search papers, labs, and topics across Lattice.
PolyBench, a new benchmark, is introduced to evaluate compositional reasoning capabilities of Large Audio Language Models (LALMs) in polyphonic audio. The benchmark includes five subsets: counting, classification, detection, concurrency, and duration estimation, all designed to test reasoning about concurrent sound events. Experiments using state-of-the-art LALMs on PolyBench reveal a significant performance drop compared to simpler audio tasks, highlighting a key limitation in current LALM architectures.
LALMs struggle with polyphonic audio, losing significant performance on tasks requiring reasoning about concurrent sound events, as revealed by the new PolyBench benchmark.
Large Audio Language Models (LALMs) are increasingly capable of reasoning over audio. However, existing benchmarks provide limited coverage of reasoning in polyphonic audio, where multiple sound events co-occur and induce compositional structure. In this work, we introduce PolyBench, a benchmark designed to evaluate compositional reasoning in polyphonic audio. PolyBench comprises five evaluation subsets covering counting, classification, detection, concurrency, and duration estimation, requiring reasoning over multiple concurrent events and their relations. Evaluation of state-of-the-art LALMs reveals consistent performance degradation in polyphonic audio, indicating a fundamental bottleneck in current LALMs.
