Search papers, labs, and topics across Lattice.
This paper introduces the Chinese Multi-Panel Meme (CMPM) benchmark, which evaluates vision-language models on their ability to understand and reason about the ordered structure of memes. The study reveals that while large vision-language models (LVLMs) perform well on single-image tasks, they struggle significantly with order-sensitive reasoning, as evidenced by a marked drop in accuracy when the visual elements are shuffled. Benchmarking five LVLMs, the results show that leading models like Gemini 3.1 Pro and GPT-5.5 outperform open models, highlighting the importance of contextual understanding in multimodal tasks.
LVLMs exhibit a dramatic decline in performance when faced with shuffled meme panels, underscoring a critical gap in order-sensitive reasoning capabilities.
Many multimodal tasks depend on how visual elements are ordered and composed, not only on recognizing them in isolation. Internet memes are a compact case of this problem: their punchline often depends on a constrained reading order and cross-panel visual--textual cues. While large vision-language models (LVLMs) show strong performance on single-image understanding, it remains unclear whether they can perform sequence-aware reasoning over structured meme layouts, especially in Chinese social media. We introduce CMPM, a Chinese Multi-Panel Meme benchmark with 1,214 annotated samples covering five structural types, ordering dependency, panel-order constraints, and optional comment context. We formulate a two-layer evaluation: Task1 probes structure typing and order-sensitive panel sequencing (with a context ablation setting), and Task2 evaluates Chinese meme explanation generation with human ratings on five 1-3 Likert dimensions (visual, panel, humor, context, and faithfulness). We benchmark five representative LVLMs under a unified protocol. Results indicate that canonical-display accuracy is not by itself evidence of order understanding: the primary shuffled condition produces a sharp accuracy drop, revealing a persistent gap in order-sensitive multimodal reasoning. Task2 preferences place Gemini 3.1 Pro and GPT-5.5 above the open models, while comment context yields only a small and mixed Core4 gain. Code and data will be released upon acceptance.
