Search papers, labs, and topics across Lattice.
The paper introduces MAGNET, a novel neural network architecture for multimodal recommendation that addresses the challenges of heterogeneous and conflicting multimodal signals by using a mixture of graph experts. MAGNET employs interaction-conditioned expert routing coupled with structure-aware graph augmentation to control and interpret multimodal fusion, using dominant, balanced, and complementary experts for behavioral, visual, and textual cues. A two-stage entropy-weighting mechanism stabilizes sparse routing and prevents expert collapse by transitioning training from coverage-oriented to specialization-oriented regimes.
Achieve more interpretable and adaptive multimodal recommendation by explicitly controlling *what* and *how* to fuse modalities with a mixture-of-experts architecture guided by interaction data and dynamically weighted by routing entropy.
Multimodal recommendation enhances ranking by integrating user-item interactions with item content, which is particularly effective under sparse feedback and long-tail distributions. However, multimodal signals are inherently heterogeneous and can conflict in specific contexts, making effective fusion both crucial and challenging. Existing approaches often rely on shared fusion pathways, leading to entangled representations and modality imbalance. To address these issues, we propose \textbf{MAGNET}, a \textbf{M}odality-Guided Mixture of \textbf{A}daptive \textbf{G}raph Experts \textbf{N}etwork with Progressive \textbf{E}ntropy-\textbf{T}riggered Routing for Multimodal Recommendation, designed to enhance controllability, stability, and interpretability in multimodal fusion. MAGNET couples interaction-conditioned expert routing with structure-aware graph augmentation, so that both \emph{what} to fuse and \emph{how} to fuse are explicitly controlled and interpretable. At the representation level, a dual-view graph learning module augments the interaction graph with content-induced edges, improving coverage for sparse and long-tail items while preserving collaborative structure via parallel encoding and lightweight fusion. At the fusion level, MAGNET employs structured experts with explicit modality roles -- dominant, balanced, and complementary -- enabling a more interpretable and adaptive combination of behavioral, visual, and textual cues. To further stabilize sparse routing and prevent expert collapse, we introduce a two-stage entropy-weighting mechanism that monitors routing entropy. This mechanism automatically transitions training from an early coverage-oriented regime to a later specialization-oriented regime, progressively balancing expert utilization and routing confidence. Extensive experiments on public benchmarks demonstrate consistent improvements over strong baselines.
