Search papers, labs, and topics across Lattice.
This paper introduces Sequential Modality Dropout (SMD), a training technique that independently erases image and text modalities during user interaction history to enhance the robustness of multi-modal sequential recommendation systems. By measuring retention, the authors demonstrate that SMD significantly improves the model's ability to maintain recommendation accuracy when modalities are missing at test time, achieving up to a 2.8x increase in retention under extreme conditions. The approach is shown to be effective across multiple backbones and domains, providing a simple yet powerful method to improve real-world applicability of recommendation systems.
SMD boosts recommendation accuracy retention by up to 2.8x when faced with missing modalities, transforming the robustness of multi-modal systems in real-world applications.
Multi-modal sequential recommenders assume every item carries every modality, but real product catalogs often miss images or text, and a model trained on complete data loses much of its recommendation accuracy when a modality is unavailable at serving time. We propose Sequential Modality Dropout (SMD): during training, each modality stream (image and text) is independently erased with probability p for an entire user interaction history, so the model learns to predict the next item without relying on any single modality. We measure robustness by retention, the fraction of a model's full-modality accuracy (HR@10) that survives when a modality is removed at test time. Across four backbones (MM-SASRec, IISAN, MISSRec, and fMRLRec) on four Amazon domains, SMD raises text retention by 1.0 to 3.2x at essentially no cost to full-modality accuracy; under an extreme 95% per-item missing rate, it retains 61% of HR@10 versus 22% without (a 2.8x improvement). An optional cross-modal reconstruction loss further lifts retention from 90% to 98% on a simple additive backbone under severe text missingness. SMD is a four-line, architecture-agnostic change that makes multi-modal sequential recommenders robust to the missing modalities they actually encounter in deployment.
