Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper details the localization of the MMLU dataset into 11 European languages through a collaboration between the DGT and the EMT, aimed at enhancing LLM evaluation benchmarks. The initiative not only fosters inclusivity in multilingual assessments but also provides master's students with hands-on experience in translation and project management. Key challenges related to methodology and workflow are discussed, emphasizing the project's dual focus on academic training and practical application.
Localizing the MMLU dataset not only enriches LLM evaluation but also transforms translation education into a collaborative, real-world experience for students.
This paper reports on a collaboration between the Directorate-General for Translation (DGT) and the European Master's in Translation (EMT) to localise the MMLU dataset into 11 European languages. Beyond creating a more inclusive benchmark for LLM evaluation, the project offers master's students authentic, project-based professional training in translation, revision, project management, and multilingual coordination, while highlighting key methodological, administrative, and workflow challenges.