Search papers, labs, and topics across Lattice.
This paper investigates the impact of multilingual post-training on LLM performance across mathematical reasoning and API calling tasks, using 220 supervised fine-tuning runs on models up to 8B parameters. The study reveals that increasing language coverage during post-training generally improves performance, particularly for low-resource languages, and that even minimal multilinguality enhances both English performance and cross-lingual generalization. Surprisingly, high language diversity in post-training can enable zero-shot cross-lingual transfer to match or exceed the performance of direct language inclusion in low-diversity settings.
English-only post-training of LLMs is suboptimal: even a single non-English language improves both English performance and cross-lingual generalization.
Despite the widespread multilingual deployment of large language models, post-training pipelines remain predominantly English-centric, contributing to performance disparities across languages. We present a systematic, controlled study of the interplay between training language coverage, model scale, and task domain, based on 220 supervised fine-tuning runs on parallel translated multilingual data mixtures spanning mathematical reasoning and API calling tasks, with models up to 8B parameters. We find that increasing language coverage during post-training is largely beneficial across tasks and model scales, with low-resource languages benefiting the most and high-resource languages plateauing rather than degrading. Even minimal multilinguality helps: incorporating a single non-English language improves both English performance and cross-lingual generalization, making English-only post-training largely suboptimal. Moreover, at sufficient language diversity, zero-shot cross-lingual transfer can match or exceed the effects of direct language inclusion in a low-diversity setting, although gains remain limited for typologically distant, low-resource languages.
