Search papers, labs, and topics across Lattice.
This study conducts a comparative analysis of 27 cross-lingual alignment (CLA) score variants to assess their predictive power for multilingual classification and translation tasks in large language models (LLMs). The authors introduce a PMI-based translation metric that minimizes dependency on target languages, revealing that CLA scores based on English outperform source-target CLA in predicting translation quality. The findings suggest that LLMs leverage English as a pivotal language for multilingual tasks, challenging existing assumptions about language independence in model performance.
CLA scores based on English can predict translation quality better than direct source-target alignment, highlighting English's role as a crucial pivot in multilingual LLMs.
Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the given language are more aligned to English within the model. Several cross-lingual alignment (CLA) scores have been proposed for use with LLMs, along with multiple approaches for extracting embeddings from the models. We provide a comparative analysis of 27 CLA score variants, examining how they differ and how well each predicts downstream performance across three tasks. Crucially, while LLMs are widely used for generative tasks such as machine translation, prior work has focused almost exclusively on classification. We therefore investigate whether CLA scores are similarly predictive of translation performance. To enable computing correlations across target languages, we propose a PMI-based translation metric, which is less dependent on the target language and correlates strongly with chrF. We find that CLA with English predicts translation quality comparably to or better than source-target CLA, providing new evidence that LLMs use English as an internal pivot language.
