Search papers, labs, and topics across Lattice.
This study investigates the performance degradation of Multilingual Large Language Models (MLLMs) when processing Code-Switched (CS) inputs, revealing a phenomenon termed Anchor Bias that measures how closely CS representations align with their source or target language counterparts. The research identifies a grammar-frame effect, where source-framed CS remains anchored to the source language, while target-framed CS shifts towards the target language, leading to significant drops in Question Answering (QA) performance. To address these challenges, the authors introduce CANVAS, an intervention that effectively aligns target-language hidden states with their source anchors, resulting in improved QA performance across various MLLMs and CS scenarios.
Code-switching in multilingual models reveals a surprising grammar-frame effect that can significantly degrade performance, but a new intervention can effectively mitigate these issues.
Multilingual Large Language Models (MLLMs) are increasingly expected to handle Code-Switched (CS) inputs, yet mixing languages frequently degrades performance relative to source- or target-language monolingual counterparts. To understand this degradation, we use grammar-forced CS as a controlled diagnostic setting for locating CS representations relative to their source and target counterparts. We introduce Anchor Bias, a geometric measure that quantifies language anchoring, whether a CS hidden state aligns closer to its source or target language counterpart. Across diverse MLLMs, Anchor Bias reveals a consistent grammar-frame effect: source-framed CS stays source-anchored, whereas target-framed CS shifts target-ward and shows larger Question Answering (QA) degradation. Motivated by this representational pattern, we propose CANVAS (Contextual Anchor-based Neural Vector Alignment Steering), an inference-time intervention that extracts a source-side canvas from the input and softly steers target-language hidden states toward the source anchor during prefill. CANVAS consistently recovers QA F1 across MLLMs and CS conditions, showing that internal anchoring signals provide an actionable target for mitigating CS inference failures.
