Search papers, labs, and topics across Lattice.
This paper addresses the limitations of existing multimodal representation frameworks in Click-Through Rate (CTR) prediction by introducing a novel Mine-Then-Train method that enhances the alignment of multimodal encoders with user click preferences. The authors identify that traditional end-to-end training fails due to ambiguous supervision stemming from the interplay of multimodal semantics and non-multimodal factors in user behavior. Through both offline and online experiments, the proposed method significantly improves CTR prediction performance by utilizing high-quality, multimodally interpretable training samples mined from raw CTR data.
Mining high-quality training samples from CTR data can drastically enhance multimodal representation learning, leading to superior prediction accuracy.
Multimodal representations have been widely adopted in industrial e-commerce recommendation systems. Due to their strong semantic understanding and generalization capabilities, they enhance the performance of traditional sparse ID-based Click-Through Rate (CTR) prediction models. Current multimodal application frameworks in the CTR prediction task typically follow a two-stage paradigm: first, pre-training a multimodal encoder on data from specific recommendation scenarios; second, extracting items'multimodal representations using this pre-trained multimodal encoder and integrating them into the CTR prediction model. However, the training objectives and data distribution of multimodal pre-training tasks often differ from those of the CTR prediction task, which limits the effectiveness of multimodal representation on downstream tasks. In this paper, we focus on how to learn Native Multimodal Representation for the CTR prediction task. One intuitive solution is to jointly train the multimodal encoder and CTR model end-to-end on the CTR task, with the expectation that the encoder can automatically learn downstream-relevant knowledge. However, we find that the end-to-end training does not bring performance improvements to existing multimodal application paradigms. Our analysis reveals that user behaviors in raw CTR data are driven by both multimodal semantics and non-multimodal factors, leading to ambiguous supervision and inconsistent encoder updates. To address this, we propose a Mine-Then-Train method that mines high-quality, multimodally interpretable training samples from CTR data and uses them to fine-tune the multimodal encoder for better alignment with user click preferences. Offline and online experiments demonstrate the effectiveness of our approach.
