Search papers, labs, and topics across Lattice.
OmniAlign is a unified multilingual aligner that effectively performs both word-level and sentence-level alignment using a single lightweight model. The approach leverages an encoder-only backbone and a four-stage training pipeline that includes alignment-oriented pre-training, self-supervised learning, supervised fine-tuning, and sentence-embedding distillation. Experiments reveal that OmniAlign not only achieves competitive performance on alignment benchmarks but also enhances alignment quality for short texts while maintaining robustness in long-context scenarios.
A single model can now achieve state-of-the-art performance in both word and sentence alignment across multiple languages, simplifying the alignment process significantly.
Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a single granularity, so practitioners often need separate systems for word- and sentence-level alignment---especially in multilingual and long-text settings. We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment with a single lightweight model. Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices, and obtains document-level $m$--$n$ sentence alignments via sentence embeddings combined with dynamic programming. To balance fine-grained alignment accuracy and sentence-representation quality, we use a four-stage training pipeline: alignment-oriented continued pre-training, self-supervised learning, supervised fine-tuning on human annotations, and sentence-embedding distillation from a strong multilingual teacher. Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks and generalizes well to unseen language pairs. Surprisingly, later-stage supervised fine-tuning on short texts further improves alignment quality while retaining the long-context understanding acquired in earlier training, keeping the model robust on long-text word alignment. \normalsize {\color{blue}\textbf{Code}: https://github.com/MilkDargon/OmniAlign}\par {\color{blue}\textbf{Model}: https://huggingface.co/WPS-Qingqiu/OmniAlign}
