Search papers, labs, and topics across Lattice.
This paper introduces ltzGLUE, the inaugural Natural Language Understanding benchmark for Luxembourgish, addressing a significant gap in NLU resources for this official national language. By constructing new tasks and adapting existing ones, the authors provide a comprehensive evaluation framework for encoder models, covering essential tasks such as named entity recognition and topic classification. The evaluation of various pre-trained models reveals the current capabilities and limitations of NLU in Luxembourgish, highlighting the need for further development in this underrepresented language.
Luxembourgish NLU is finally getting the attention it deserves with the launch of ltzGLUE, revealing critical insights into model performance and language capabilities.
This paper presents ltzGLUE, the first Natural Language Understanding (NLU) benchmark for Luxembourgish (LTZ) based on the popular GLUE benchmark for English. Although NLU tasks are available for many European languages nowadays, LTZ is one of the official national languages that is often overlooked. We construct new tasks and reuse existing ones to introduce the first official NLU benchmark and accompanying evaluation of encoder models for the language. Our tasks include common natural language processing tasks in binary and multi-class classification settings, including named entity recognition, topic classification, and intent classification. We evaluate various pre-trained language models for LTZ to present an overview of the current capabilities of these models on the LTZ language.
