Search papers, labs, and topics across Lattice.
This study addresses the challenge of generating dialectally accurate outputs in Arabic NLP by analyzing the internal representations of Arabic LLMs. By identifying sparse neuron populations that encode dialect-specific features, the authors demonstrate that manipulating these neurons can effectively steer model outputs toward desired dialects. Additionally, they introduce a vector-steering approach that utilizes extracted activation directions to enhance dialect control during inference, providing a novel framework for improving dialect generation without the need for fine-tuning.
Manipulating just a few neurons can transform Arabic LLM outputs from Modern Standard Arabic to specific dialects, revealing a surprising level of control over dialectal generation.
A key challenge in Arabic NLP is the scarcity of dialectal data relative to Modern Standard Arabic (MSA), causing LLMs to overproduce MSA and struggle with dialectally accurate generation. From an interpretability perspective, this raises a fundamental question: where and how are dialectal features encoded within model internals, and can these representations be leveraged to improve dialect generation without fine-tuning? This study investigates two complementary inference-time approaches that serve simultaneously as interpretability probes and control mechanisms. First, we conduct a neuron-level analysis, identifying sparse neuron populations that encode dialect-specific features and showing that amplifying or suppressing these neurons can steer model outputs toward target dialects. Second, motivated by the entanglement of dialectal features at the single-neuron level, we apply a vector-steering approach that extracts dialect-specific activation directions and injects them during inference. Together, these methods illuminate the geometry of dialectal knowledge in Arabic LLMs and offer a principled, interpretability-grounded framework for dialect control without requiring dialect-specific fine-tuning.
