Search papers, labs, and topics across Lattice.
This study investigates the shared internal mechanisms of multilingual large language models (LLMs) in the context of present-tense subject-verb agreement across 29 languages. By employing activation patching and attention analysis, the authors reveal that languages with overt person/number inflection exhibit more similar agreement circuitry compared to non-conjugating languages, with significant overlap occurring when focusing on the recovery of inflectional contrasts. The findings suggest that multilingual LLMs leverage partially shared computational structures for morphosyntactic agreement, challenging the notion of entirely separate language-specific solutions.
Languages with overt inflection share more agreement circuitry, revealing that multilingual LLMs reuse computational structures instead of relying on distinct solutions for each language.
Multilingual large language models often generalize across languages, and prior work suggests that their internal mechanisms can overlap cross-lingually. It remains unclear, however, when such sharing emerges and whether it varies with the overt realization of the same grammatical operation. We investigate this question for present-tense subject-verb agreement, a morphosyntactic process that varies substantially across languages and is only weakly expressed in English. Using activation patching and attention analysis across 29 languages and five open-source model families, we identify the attention heads causally implicated in agreement and compare these head-level signatures across languages. We find that languages with overt person/number inflection exhibit more similar agreement circuitry than non-conjugating languages, with the strongest sharing appearing when the analysis isolates recovery of the inflectional contrast itself. English provides an informative bridge case, becoming more similar to conjugating languages precisely in contexts where overt agreement is required. Finally, many implicated heads display similar attention patterns across languages, suggesting that cross-lingual overlap reflects shared functional roles as well as shared localization. Together, these results indicate that multilingual LLMs reuse partially shared computational structure for morphosyntactic agreement rather than relying on fully separate language-specific solutions.
