Search papers, labs, and topics across Lattice.
This paper introduces LatentMT, a novel approach to machine translation that utilizes latent-reasoning looped language models (LoopLMs) to enhance translation quality without the need for larger parameter counts. By leveraging a 2.6B-parameter backbone model and incorporating recurrent computation within hidden states, LatentMT achieves competitive performance across 32 translation directions, particularly excelling in mid- and low-resource languages. The findings reveal that while increasing the number of reasoning steps improves translation quality initially, it quickly reaches a saturation point, highlighting the efficiency of this method compared to larger models.
LatentMT achieves state-of-the-art translation performance on low-resource languages using a compact model that is three to five times smaller than its competitors.
Latent-reasoning looped language models (LoopLMs) offer a different scaling path for machine translation (MT): instead of increasing parameter count or emitting explicit chain-of-thought tokens, they spend additional recurrent computation inside hidden states. We introduce LatentMT, the first systematic study of latent-reasoning LoopLMs for machine translation. LatentMT adapts a small 2.6B-parameter backbone model with lightweight training. Across 32 translation directions spanning high-, mid-, and low-resource languages, LatentMT achieves performance comparable to models three to five times larger. It is competitive in a high-resource language and achieves state-of-the-art performance on both mid-resource and low-resource languages. Studying the behavior of scaling the number of recurrent reasoning steps, we find that recurrent computation consistently improves translation quality in early steps, then saturates quickly afterwards. Our mechanistic analysis shows that hidden-representation differences shrink along the recurrent reasoning-step axis, supporting the observed saturation in performance. Finally, our efficiency analysis shows that LatentMT requires lower training and inference compute than much larger non-latent-reasoning models with similar performance, making latent recurrent computation a promising path toward compact, efficient, and strong machine translation.
