Search papers, labs, and topics across Lattice.
To address English-centric biases in speech foundation models, the authors built SEA-SpeechBench, an evaluation suite covering 11 Southeast Asian languages across 597 audio hours and 97k samples spanning core speech processing, paralinguistics, and long-form temporal localization up to 3 minutes. Evaluating frontier open-source and proprietary audio-LLMs reveals pervasive blind spots in temporal grounding, speech translation, and emotion recognition. Crucially, prompting models in native low-resource languages like Burmese and Tamil yields severe performance degradation, lagging behind English equivalents by up to 41 percentage points.
Leading audio LLMs suffer up to a 41-percentage-point performance drop when prompted in native Southeast Asian languages compared to English, exposing severe blind spots in acoustic temporal grounding and paralinguistics.
The rapid advancement of audio and multimodal large language models has unlocked transformative speech understanding capabilities, yet evaluation frameworks remain predominantly English-centric, leaving Southeast Asian (SEA) languages critically underrepresented. We introduce SEA-SpeechBench, to the best of our knowledge, the first large-scale multitask benchmark that evaluates speech understanding in 11 SEA languages through 97,194 samples across 99 evaluation sets and 597 hours of curated audio data. Our benchmark comprises 9 diverse tasks across 3 categories: speech processing (automatic speech recognition, speech translation, spoken question answering), paralinguistic analysis (emotion, gender, age, speaker recognition), and temporal understanding, a novel dimension featuring timestamped content queries and temporal localization within extended audio sequences up to 3 minutes. We implement multilingual prompting in both native SEA languages and English to reflect user interactions with audio-language models. Evaluation of leading open-source and proprietary systems reveals marked performance gaps. Across all models, performance remains underwhelming on temporal understanding, emotion recognition, and speech translation. Prompting in low-resource languages such as Burmese and Tamil lags behind English by up to 41 percentage points. Our findings expose critical model limitations and underscore the need for inclusive model development. The SEA-SpeechBench benchmark is available at https://zwenyu.github.io/SEA-SpeechBench/.
