Search papers, labs, and topics across Lattice.
This paper investigates a unified multi-task relevance model for e-commerce that addresses six different entity pair relationship types, revealing the limitations of current practices that rely on separate models. By systematically comparing three task routing architectures across both LoRA adapted LLMs and fully fine-tuned cross-encoders, the authors demonstrate that the MHP Ensemble achieves the highest accuracy at 89.96% on a large dataset. Furthermore, the study highlights an encoder-decoder asymmetry in task identity encoding, showing that multi-task training can significantly enhance performance on low-resource tasks.
The MHP Ensemble not only outperforms existing models but also reveals critical insights into how task routing architectures impact performance across different model types.
How can we build a single relevance model that handles six different entity pair relationship types in e commerce from query product matching to product type similarity when each task has different data volumes, different semantic requirements, and potentially conflicting learning signals? This question is important because current industry practice relies on separate models for each task, preventing knowledge transfer and producing inconsistent relevance signals. Our work is driven by the following insight: encoder based and decoder only models encode task identity through different mechanisms, so the choice of task routing architecture how task identity is communicated to the shared model affects these two families in asymmetric ways. As our key novelty, we combine three ideas: (a) a unified multi task framework that jointly trains on six entity pair tasks under a shared three point relevance scale, (b) a systematic comparison of three task routing architectures (text prefix routing, multi head classification, and multihead with private transformer layers) across both LoRA adapted LLMs and fully finetuned cross encoders, and (c) a majority vote ensemble that exploits the diversity induced by private layer routing. First, we show that the MHP Ensemble (multi head with private layers) achieves 89.96% accuracy on 453K test examples the highest across all configurations . Second, we show that removing text prefixes without private layers causes severe degradation for decoder only LLMs while cross encoders remain robust , suggesting an encoder decoder asymmetry in task identity encoding. Third, we show that multi task training yields up to 14% improvement on low resource tasks over single task baselines.
