Search papers, labs, and topics across Lattice.
This paper introduces a novel framework for multimodal sarcasm detection that combines Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization (SaCR) to address the challenges of instance-dependent modality contributions and misleading semantic consistency. By employing a bidirectional gated interaction module, the framework adaptively filters and calibrates textual and visual inputs, while SaCR enhances the model's ability to distinguish between sarcastic and non-sarcastic content. Experimental results on the MMSD and MMSD2.0 datasets show significant improvements over existing methods, highlighting the effectiveness of the proposed approach in capturing subtle sarcasm cues.
Sarcastic-aware contrastive regularization enables the model to discern nuanced sarcasm, outperforming traditional methods that struggle with modality inconsistencies.
Multimodal sarcasm detection aims to identify sarcastic intent from multimodal content, where inconsistencies between literal meaning and contextual cues often signal irony. This task has attracted increasing research attention. However, accurate detection remains challenging due to instance-dependent modality contributions and misleading semantic consistency, where surface-level alignment masks underlying contradictory intent. Existing methods often rely on fixed fusion strategies and treat sarcasm as generic cross-modal mismatch, limiting their ability to capture subtle sarcasm cues and instance-specific modality interactions. To address these challenges, we propose a novel MSD framework that integrates Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization (SaCR). Specifically, a bidirectional gated interaction module performs cross-modal feature filtering and adaptively calibrates textual and visual contributions at the instance level. A dynamic fusion gate further balances modality importance to generate more robust multimodal representations. Furthermore, SaCR is introduced as a label-aware contrastive regularization objective that encourages semantic consistency for non-sarcastic samples while suppressing misleading consistency in sarcastic cases. The proposed framework is trained end-to-end with a multi-objective learning strategy that jointly optimizes multimodal classification and auxiliary unimodal supervision. Extensive experiments on MMSD and MMSD2.0 demonstrate that the proposed method consistently outperforms strong baselines.
