Search papers, labs, and topics across Lattice.
This paper introduces MMDS-Bench, a benchmark designed for evaluating multimodal dynamic stance classification in social media interactions, which considers how replies relate to their parent messages in a dynamic context. The benchmark includes 3,482 annotated instances with a seven-label stance taxonomy and a diagnostic subset that emphasizes structured reasoning across various challenge factors. Evaluation of 12 multimodal large language models reveals significant shortcomings in their ability to understand complex relational dynamics, highlighting the need for improved models in this area.
Current multimodal large language models struggle with dynamic stance understanding, particularly when relational inference is required, revealing critical gaps in their reasoning capabilities.
Dynamic stance classification models how a reply responds to its direct parent message, rather than how a post relates to a fixed topic. Existing work has mainly studied this problem in text-only settings, while social media interactions increasingly rely on images, screenshots, memes, reaction images, and cross-modal references. We introduce MMDS-Bench, a diagnostic benchmark for multimodal dynamic stance classification in social media parent-reply interactions. MMDS-Bench contains 3,482 multimodal instances annotated with a seven-label dynamic stance taxonomy, together with an 800-instance diagnostic subset that requires structured reasoning over parent understanding, reply understanding, and stance-relation inference. We further annotate each instance with five challenge factors covering multimodal fusion, parent framing, non-literal expression, interaction reasoning, and label-boundary ambiguity. We evaluate 12 closed-source and open-source multimodal large language models and propose a reference-grounded LLM-judge protocol for assessing reasoning quality. Results show that current MLLMs still struggle with multimodal dynamic stance understanding, especially in cases that require relational inference beyond separate parent and reply comprehension.
