Search papers, labs, and topics across Lattice.
The paper investigates the ability of large language models (LLMs) to converse in Tulu, an extremely low-resource language, using only structured prompting without fine-tuning. They address the lack of Tulu training data by employing explicit grammar documentation, negative constraints to reduce vocabulary contamination, romanization standardization, and quality-controlled synthetic data generation. Experiments on Gemini 2.0 Flash, GPT-4o, and Llama 3.1 70B, validated by native speakers, demonstrate a reduction in vocabulary contamination from 80% to 5% and achieve 85% grammatical accuracy, highlighting the effectiveness of structured prompting for low-resource languages.
LLMs can achieve 85% grammatical accuracy in a language with minimal training data, like Tulu, using only structured prompting and clever constraints.
Can large language models converse in languages virtually absent from their training data? We investigate this question through a case study on Tulu, a Dravidian language with over 2 million speakers but minimal digital presence. Rather than fine-tuning an LLM, we examine whether structured prompts alone can elicit basic conversational ability under controlled prompting. We systematically tackle various challenges posed by absence of training data for Tulu by combining explicit grammar documentation, negative constraints to suppress high-probability tokens from related languages, romanization standardization, and quality-controlled synthetic data generation via self-play. Evaluated on a manually curated held-out set across three LLMs (Gemini 2.0 Flash, GPT-4o, Llama 3.1 70B) and validated by native speakers, our approach reduces vocabulary contamination from 80% to 5% while achieving 85% grammatical accuracy. Cross-model analysis reveals that negative constraints provide consistent improvements (12--18 percentage points), while grammar documentation effects vary by model architecture (8--22 points).
