Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper addresses the limitations of existing multimodal representation frameworks in Click-Through Rate (CTR) prediction by introducing a Mine-Then-Train method that refines the training process. By mining high-quality, multimodally interpretable samples from raw CTR data, the authors enhance the alignment of the multimodal encoder with user click preferences, overcoming the challenges posed by ambiguous supervision in traditional end-to-end training. Experimental results show significant improvements in CTR prediction performance, highlighting the importance of tailored training data in multimodal applications.
Mining high-quality samples from CTR data can dramatically enhance multimodal representation learning, leading to superior prediction accuracy.
Multimodal representations have been widely adopted in industrial e-commerce recommendation systems. Due to their strong semantic understanding and generalization capabilities, they enhance the performance of traditional sparse ID-based Click-Through Rate (CTR) prediction models. Current multimodal application frameworks in the CTR prediction task typically follow a two-stage paradigm: first, pre-training a multimodal encoder on data from specific recommendation scenarios; second, extracting items'multimodal representations using this pre-trained multimodal encoder and integrating them into the CTR prediction model. However, the training objectives and data distribution of multimodal pre-training tasks often differ from those of the CTR prediction task, which limits the effectiveness of multimodal representation on downstream tasks. In this paper, we focus on how to learn Native Multimodal Representation for the CTR prediction task. One intuitive solution is to jointly train the multimodal encoder and CTR model end-to-end on the CTR task, with the expectation that the encoder can automatically learn downstream-relevant knowledge. However, we find that the end-to-end training does not bring performance improvements to existing multimodal application paradigms. Our analysis reveals that user behaviors in raw CTR data are driven by both multimodal semantics and non-multimodal factors, leading to ambiguous supervision and inconsistent encoder updates. To address this, we propose a Mine-Then-Train method that mines high-quality, multimodally interpretable training samples from CTR data and uses them to fine-tune the multimodal encoder for better alignment with user click preferences. Offline and online experiments demonstrate the effectiveness of our approach.