Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the first Komi-Yazva--Russian parallel corpus, consisting of 457 aligned sentence pairs from 74 narrative texts, aimed at evaluating LLM translation in a low-resource context. The authors establish a comprehensive evaluation protocol that includes story-level cross-validation and various metrics to assess translation performance under zero-shot and few-shot conditions. Results indicate that while LLMs can generate meaningful translations, their effectiveness is highly dependent on the model family and prompting strategy, highlighting the importance of metric choice in evaluation outcomes.
LLMs can translate endangered languages, but their performance hinges critically on the choice of prompting strategy and evaluation metrics.
We present the first Komi-Yazva--Russian parallel corpus together with an explicit evaluation protocol for studying LLM translation in an endangered, extremely low-resource setting. The dataset contains 457 aligned sentence pairs from 74 narrative texts and is accompanied by documented provenance, sentence-level alignment, and story identifiers that enable leakage-aware evaluation. We use this setup to compare modern large language models on Komi-Yazva-to-Russian translation under severe parallel-data scarcity in zero-shot and retrieval-based few-shot regimes. The protocol includes story-level cross-validation, deterministic retrieval for few-shot prompting, strict validation of generated outputs, complementary reference-based and judge-based metrics, and story-level uncertainty estimates. Across models, LLMs produce non-trivial translations, but performance varies strongly by model family and prompting regime. Retrieval-based few-shot prompting consistently improves over zero-shot prompting, while gains beyond a small retrieved context remain limited. The results show that evaluative conclusions in this setting depend materially on metric choice and failure handling, so the paper frames the corpus as both a dataset contribution and a reproducible evaluation testbed for endangered-language machine translation.