Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces DeltaMerge-LowRes, a method for adapting multilingual encoders to new languages and tasks using separate training of language and task deltas. By employing a novel cross-axis TIES merging rule, the approach significantly enhances performance in low-resource settings, achieving notable improvements in summarization and question answering across multiple African languages. The results indicate that the method not only boosts accuracy but also improves calibration in classification tasks, thereby changing the dynamics of how language and task adaptations are combined.
Cross-axis TIES merging outperforms traditional methods, boosting summarization scores by up to 7 chrF points across multiple languages.
Adapting a multilingual encoder to a new language \emph{and} a new task with only a few hundred gold examples is a common low-resource NLP setting, yet the two axes are usually fused via an expensive language--task fine-tuning run. We ask whether they can instead be trained separately and recombined in weight space. \DeltaMergeLowRes{} learns a language delta $\Delta_L$ from unlabeled monolingual text and a task delta $\Delta_T$ from labeled English data, then composes them at inference under one of four rules: additive, activation-guided, sparsity-aware, and a novel \emph{cross-axis TIES}. The new rule adapts the TIES-Merging steps of trimming, sign election, and merging to the language and task axes rather than to two task axes. Holding $(\Delta_L,\Delta_T)$ fixed across rules on four task families and four African languages ($158$ evaluated cells, $10{,}000$-sample paired bootstrap per cell), we find: (i) cross-axis TIES wins summarisation on $3/4$ languages by $+4$ to $+7$ chrF (chrF $18.59$ vs.\ $13.80$ task-only); (ii) it improves QA F1 by $+2.32$ and EM by $+2.91$; and (iii) sparsity-aware merging cuts classification ECE by $36\%$ at parity macro-F1. The composition rule materially changes what the merged model preserves, suppresses, and calibrates. We release all JSON traces and a claim ledger.