Search papers, labs, and topics across Lattice.
This study evaluates the dialectal robustness of large language models (LLMs) and speech language models (SLMs) using Japanese dialects, focusing on how dialect comprehension impacts performance. By defining robustness as the performance ratio on dialectal versus standard inputs, the authors provide a framework for fair comparisons between model types. Key findings reveal that SLM robustness is closely linked to the robustness of their text-based counterparts, with enhancements achieved through dialectal training and fine-tuning of the speech encoder.
SLMs show improved dialectal understanding when trained on dialectal data, revealing a critical pathway for enhancing performance in diverse linguistic contexts.
Dialogue systems based on large language models (LLMs) have advanced significantly in recent years. However, dialectal variation remains a major challenge, particularly for systems that process spoken input. LLM-based speech language models (SLMs), which integrate LLMs with speech processing components, show promise for spoken language tasks, yet their ability to comprehend dialects has not been sufficiently studied. Moreover, it remains unclear how the dialectal understanding of the base LLM affects SLM performance. This study investigates the dialectal robustness of both LLMs and SLMs using Japanese dialects as a test case. We define robustness as the ratio of performance on dialectal versus standard inputs, enabling fair comparisons. Our experiments show that SLM robustness correlates with that of their text-based counterparts. Furthermore, training with dialectal data and fine-tuning the speech encoder each improves robustness in SLMs.
