Search papers, labs, and topics across Lattice.
This paper investigates the transferability of Tiny Recursive Models (TRM) to Quality Estimation (QE) for low-resource languages. Through experiments on 8 language pairs, the authors find that TRM's recursive mechanisms do not readily transfer to QE tasks, with external iteration harming performance and internal recursion offering limited benefits. However, they demonstrate that using frozen, pretrained XLM-R embeddings in a TRM-QE architecture achieves comparable or superior performance to fine-tuned models and standard transformers, while drastically reducing the number of trainable parameters.
Freezing pretrained embeddings in a Tiny Recursive Model slashes trainable parameters by 37x while matching or exceeding the quality estimation performance of fine-tuned models.
Tiny Recursive Models (TRM) achieve strong results on reasoning tasks through iterative refinement of a shared network. We investigate whether these recursive mechanisms transfer to Quality Estimation (QE) for low-resource languages using a three-phase methodology. Experiments on $8$ language pairs on a low-resource QE dataset reveal three findings. First, TRM's recursive mechanisms do not transfer to QE. External iteration hurts performance, and internal recursion offers only narrow benefits. Next, representation quality dominates architectural choices, and lastly, frozen pretrained embeddings match fine-tuned performance while reducing trainable parameters by 37$\times$ (7M vs 262M). TRM-QE with frozen XLM-R embeddings achieves a Spearman's correlation of 0.370, matching fine-tuned variants (0.369) and outperforming an equivalent-depth standard transformer (0.336). On Hindi and Tamil, frozen TRM-QE outperforms MonoTransQuest (560M parameters) with 80$\times$ fewer trainable parameters, suggesting that weight sharing combined with frozen embeddings enables parameter efficiency for QE. We release the code publicly for further research. Code is available at https://github.com/surrey-nlp/TRMQE.
