Search papers, labs, and topics across Lattice.
This paper compares Continuous Chain-of-Thought (CoT) reasoning using the CODI framework against standard supervised fine-tuning for multilingual reasoning across five languages. The study finds that continuous reasoning significantly outperforms explicit reasoning, especially in zero-shot settings for low-resource languages. The approach also achieves substantial compression of reasoning traces (29x-50x), suggesting greater language invariance in continuous latent representations.
Continuous reasoning in latent space crushes explicit reasoning for multilingual tasks, especially when training data is scarce.
We investigate whether performing reasoning in a continuous latent space leads to more robust multilingual capabilities. We compare Continuous Chain-of-Thought (using the CODI framework) against standard supervised fine-tuning across five typologically diverse languages: English, Chinese, German, French, and Urdu. Our experiments on GSM8k and CommonsenseQA demonstrate that continuous reasoning significantly outperforms explicit reasoning on low-resource languages, particularly in zero-shot settings where the target language was not seen during training. Additionally, this approach achieves extreme efficiency, compressing reasoning traces by approximately $29\times$ to $50\times$. These findings indicate that continuous latent representations naturally exhibit greater language invariance, offering a scalable solution for cross-lingual reasoning.
