Search papers, labs, and topics across Lattice.
To address the objective mismatch when using cross-lingual sentence encoders for fine-grained tasks like hallucination detection, this work evaluates SALT, a lightweight post-training method that injects span-level supervision into existing encoders. Evaluated across five multilingual token-level benchmarks, the approach establishes state-of-the-art performance on four, surpassing standard fine-tuning baselines and competitive encoders. Crucially, sub-sentence alignment does not compromise global semantic structure, simultaneously boosting performance on cross-lingual sentence retrieval and classification tasks.
Span-level supervision fixes the token-representation mismatch in cross-lingual sentence encoders without sacrificing鈥攁nd in fact improving鈥攕entence retrieval performance.
Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.
