Search papers, labs, and topics across Lattice.
The authors introduce TW-Sound580K, a Taiwanese audio-text instruction dataset created using a Verify-Generate-Critique (VGC) protocol with Dual-ASR validation to ensure high fidelity. They then fine-tune a DeSTA 2.5-Audio-initialized LALM, named Tai-LALM, using this dataset and a dynamic Dual-ASR Arbitration strategy during inference. Tai-LALM achieves a 49.1% accuracy on the TAU Benchmark, a 6.5% improvement over the zero-shot baseline, demonstrating the effectiveness of regional corpora and dynamic arbitration for localized speech tasks.
Overcome LALM's struggles with localized dialectal prosody: a new Taiwanese audio-text dataset and fine-tuning strategy boosts accuracy by 6.5% on the TAU Benchmark.
Large Audio-Language Models (LALMs) typically struggle with localized dialectal prosody due to the scarcity of specialized corpora. We present TW-Sound580K, a Taiwanese audio-text instruction dataset developed through a Verify-Generate-Critique (VGC) protocol. This pipeline leverages Dual-ASR validation to filter 522K raw clips, subsequently expanding them into 580,000 high-fidelity instruction pairs using a teacher model. The dataset's utility is demonstrated through Tai-LALM, which fine-tunes a DeSTA 2.5-Audio-initialized backbone and incorporates a dynamic Dual-ASR Arbitration strategy to optimize transcription selection during inference. On the TAU Benchmark, Tai-LALM reaches 49.1% accuracy, marking a 6.5% absolute improvement over the zero-shot baseline (42.6% with ASR text conditioning). This confirms that integrating regional corpora with rigorous curation and dynamic arbitration significantly enhances LALM performance on localized speech.
