Search papers, labs, and topics across Lattice.
This paper introduces AudioScape-TTA, a structured benchmark designed to enhance the evaluation of text-to-audio (TTA) generation by focusing on fine-grained semantic accuracy. By employing modality-aware semantic structures and assessing generation complexity through event density and structural complexity, the authors provide a comprehensive framework for evaluating TTA systems. Experiments reveal that existing models struggle with fine-grained control over audio attributes and speech content, highlighting the benchmark's ability to align more closely with human judgments compared to traditional metrics.
Fine-grained evaluation reveals that current text-to-audio models fail to preserve speech content and control audio attributes effectively.
Text-to-audio (TTA) generation has recently achieved remarkable progress in synthesizing realistic audio from natural language descriptions. However, determining whether generated audio faithfully satisfies complex textual instructions remains challenging. Existing benchmarks mainly rely on global similarity metrics, providing limited insight into fine-grained semantic failures. To address this limitation, we introduce \textbf{AudioScape-TTA}, a structured and complexity-aware benchmark for fine-grained TTA evaluation. AudioScape-TTA represents realistic soundscapes through modality-aware semantic structures and characterizes generation complexity using event density and structural complexity. Based on these annotations, we propose a rubric-based audio-grounded evaluation framework that verifies event realization, acoustic attributes, and speech content through fine-grained semantic criteria. The benchmark contains 2,258 audio-text pairs with 25,707 binary QA rubrics, enabling scalable and interpretable analysis of TTA systems. Experiments on 13 representative open-source TTA models reveal persistent limitations in fine-grained attribute control, speech-content preservation, and compositional soundscape generation. Human validation further demonstrates that our rubric-based evaluation achieves stronger alignment with human semantic judgments than conventional global similarity metrics.
