Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
This study investigates the reasoning performance disparity between high-resource languages (HRLs) and low-resource languages (LRLs) in large language models, proposing that HRLs may better activate latent computations necessary for task-specific reasoning. By employing a mechanistic intervention framework that utilizes sparse autoencoders to identify and transfer task-relevant features from HRLs to LRLs, the authors demonstrate that activating these features during LRL inference can significantly enhance reasoning performance. The findings suggest that many observed cross-lingual reasoning gaps stem from differences in feature elicitation rather than inherent capabilities, offering a novel approach to improve LRL performance without traditional methods like translation or fine-tuning.
Activating task-relevant latent features from high-resource languages can significantly boost reasoning performance in low-resource languages, challenging assumptions about language capabilities.
Large language models exhibit substantial performance variation across languages, even when solving semantically equivalent tasks. Existing analyses often treat this phenomenon as an observational disparity caused by differences in pretraining data, tokenization, or benchmark coverage. We study a complementary hypothesis: high-resource languages (HRLs) may more reliably elicit latent computations useful for task-specific (i.e. mathematical) reasoning, while lower-resource languages (LRLs) may under-activate those computations despite expressing the same task. To test this hypothesis, we introduce a mechanistic intervention framework for identifying and transferring task-relevant sparse latent features across languages. Using sparse autoencoders over residual-stream activations, we isolate features enriched in successful HRL task-specific reasoning while filtering out source-language and generic-generation features. We then construct steering directions from these features and inject them during LRL inference. The resulting interventions test whether the selected features are functionally involved in the observed reasoning gap: suppressing them should impair source-language reasoning, while activating them should partially recover target-language reasoning beyond random and non-task controls. Our framework reframes some cross-lingual reasoning gaps as failures of mechanism elicitation rather than capability absence, and offers a causally testable route to feature-mediated transfer without translation, fine-tuning, or changing the user-facing language.