Search papers, labs, and topics across Lattice.
Arabic-DeepSeek-R1, a new open-source Arabic LLM, achieves SOTA on the Open Arabic LLM Leaderboard (OALL) by using a sparse MoE backbone and a four-phase Chain-of-Thought (CoT) distillation scheme incorporating Arabic-specific linguistic verification and regional ethical norms. The model outperforms GPT-5.1 on several benchmarks, demonstrating the effectiveness of specialized adaptation for Arabic. This suggests that under-specialization, rather than architectural limitations, is a key factor in the performance gap for Arabic LLMs.
Open-source Arabic-DeepSeek-R1 dethrones GPT-5.1 on Arabic language benchmarks, proving that culturally-informed adaptation of sparse MoE models can leapfrog proprietary giants without massive pretraining costs.
This paper introduces Arabic-DeepSeek-R1, an application-driven open-source Arabic LLM that leverages a sparse MoE backbone to address the digital equity gap for under-represented languages, and establishes a new SOTA across the entire Open Arabic LLM Leaderboard (OALL). Our four-phase CoT distillation scheme integrates Arabic-specific linguistic verification and regional ethical norms into a 372M-token, contamination-controlled 80/20 Arabic-English training mixture. Arabic-DeepSeek-R1 achieves the highest average score across the seven-benchmark OALL suite while establishing SOTA or near-SOTA, including dominant results on grammar-focused MadinahQA (surpassing both GPT-5.1 and the OALL leader by substantial margins), safety-oriented AraTrust, multi-ability AlGhafa, and retrieval-augmented ALRAGE. Our results indicate that the combination of sparse MoE architecture, culturally-informed CoT distillation with explicit Arabic linguistic checks, and strategic bilingual data curation enables an open-source adapted model to systematically outperform the proprietary frontier system GPT-5.1 on the majority of benchmarks evaluating comprehensive language-specific tasks: the first such demonstration for Arabic LLMs. These findings indicate that much of Arabic's performance deficit in current LLM ecosystems stems from under-specialization rather than architectural limitations, and that parameter-efficient adaptation of open reasoning models can yield breakthrough SOTA performance without industrial-scale pretraining costs. Arabic-DeepSeek-R1 establishes a validated and replicable framework for sovereign and domain-specific language technologies, demonstrating that strategic, culturally-grounded adaptation of sparse MoE backbones offers a viable and cost-effective pathway to achieving record-breaking performance across standardized benchmarks for low-resource languages.
