Search papers, labs, and topics across Lattice.
This paper introduces SAGA, a parser-guided preference optimization framework that enhances the grammatical quality of low-resource Nordic language models without relying on human preference annotations. By converting dependency-parser judgments into preference pairs and employing a composite reward system, SAGA effectively filters low-information pairs and monitors for reward hacking. The approach yields significant improvements in parse success rates and native speaker preferences across Danish, Icelandic, and Norwegian Bokm氓l, demonstrating the viability of parser-derived supervision in low-resource settings.
Parser-derived supervision can replace costly human annotations, achieving up to 93.8% parse success in Danish and 80% preference in native speaker comparisons.
Preference optimisation has proven effective for improving large language models but typically relies on costly human preference annotations. Extending these methods to morphologically rich, low-resource languages remains challenging because such annotations are scarce. We present SAGA (Score-weighted Adaptive Generation Alignment), a parser-guided preference optimisation framework that replaces human labels with dependency-parser supervision. SAGA converts parser judgements into preference pairs for delta-DPO, combines parser quality with lexical diversity in a composite reward, filters low-information pairs using a reward-gap criterion, and monitors reward hacking to maintain reliable supervision. Across Danish, Icelandic, and Norwegian Bokm{\aa}l using GPT-SW3-1.3B, SAGA consistently improves grammatical quality without requiring human preference labels. Danish parse success increases from 69.0% to 93.8%, Icelandic achieves a +4.5 percentage-point improvement on an independent Stanza evaluation (three-run mean +3.3 percentage points) while native speakers prefer SAGA outputs in 80% of pairwise comparisons, and Norwegian Bokm{\aa}l improves by +28 percentage points. These results demonstrate that parser-derived supervision is a practical alternative to human preference annotation for grammatical alignment in low-resource languages where high-quality dependency parsers are available.
