Search papers, labs, and topics across Lattice.
This paper introduces TAG-Bench, a comprehensive benchmark designed to evaluate the temporal audio grounding capabilities of large audio language models (LALMs) by assessing their ability to return accurate time intervals for natural-language queries. The benchmark comprises 1,750 human-verified query-recording pairs, covering a wide range of audio durations and query categories, revealing significant performance gaps across 21 evaluated systems, with the best model achieving only 31.2 mIoU. Notably, the findings highlight that even top-performing models struggle with recall and count accuracy, particularly on complex queries, underscoring the need for improved methods in temporal audio grounding.
Despite advances in large audio language models, the best performer in temporal audio grounding only achieves a mere 31.2 mIoU, revealing critical limitations in current capabilities.
Large audio language models (LALMs) can describe what is heard, but their ability to localize when queried content occurs remains less systematically evaluated. We present TAG-Bench, a benchmark for temporal audio grounding in which a model returns every time interval that matches a natural-language query. TAG-Bench contains 1,750 human-verified query-recording pairs covering 149.5 hours, with eight source-dependent subsets spanning query categories and audio durations from 7 s to 20 min; 22.1% of the queries have multiple ground-truth intervals. Across 21 evaluated systems, the best-performing model achieves 31.2 mIoU and is the only system above 20 mIoU on the two long subsets, yet even this top performer reaches only 21.5% recall at IoU>= 0.7. Moreover, 9 of 21 systems fall below 5 mIoU, and every model under-reports the number of occurrences on one-to-many queries, with none exceeding 13.2% count accuracy. Because responses are free-form, we report parsing-failure rate and MAE coverage: parsing failures remain in mIoU, Recall, gIoU, and count metrics as empty predictions but do not enter MAE. The results separate precise localization, occurrence enumeration, and output-format reliability within a benchmark whose cross-subset comparisons are descriptive rather than controlled estimates of query abstraction or duration. We will release the TAG-Bench data and evaluation code to support future research.
