Search papers, labs, and topics across Lattice.
This paper investigates the impact of joint vs. disjoint vocabularies on knowledge transfer in multilingual neural machine translation (MNMT), focusing on out-of-domain translation scenarios. Through systematic experiments with related and unrelated auxiliary languages, the authors find that while vocabulary overlap improves performance, domain match and language relatedness are more critical factors for effective knowledge transfer. The study highlights the nuanced interplay between vocabulary sharing and linguistic similarity in MNMT.
Domain match and language relatedness trump joint vocabularies for effective knowledge transfer in multilingual NMT.
Knowledge transfer, especially across related languages, has been found beneficial for multilingual neural machine translation (MNMT), but some aspects are still under-explored and deserve further investigation. A joint vocabulary is most often applied to form a uniform word embedding space, but since the impact of a disjoint vocabulary on model performance is far less studied, there is no consensus on how much knowledge transfer is mainly due to vocabulary overlap. In this paper, we present systematic experiments with joint and disjoint vocabularies, and auxiliary languages related and unrelated to the source language. We design this experiment in an out-of-domain setup in order to emphasize transfer and the impact of the auxiliary language. As expected, we yield better results with more extensive vocabulary overlaps typical for related languages, but our experiments also show that domain-match and language relatedness are more important than a joint vocabulary.
