Search papers, labs, and topics across Lattice.
The paper introduces CROSS-ALIGN+, a three-stage framework for meme-based social abuse detection that addresses cultural blindness, boundary ambiguity, and lack of interpretability in existing methods. CROSS-ALIGN+ enriches multimodal representations with structured knowledge, reduces boundary ambiguity using LoRA adapters, and enhances interpretability through cascaded explanations. Experiments on five benchmarks and eight LVLMs show that CROSS-ALIGN+ outperforms state-of-the-art methods, achieving up to a 17% relative F1 improvement.
You can now detect harmful memes with 17% better accuracy and understand *why* they're toxic, thanks to a new framework that injects cultural context and explains its reasoning.
Meme-based social abuse detection is challenging because harmful intent often relies on implicit cultural symbolism and subtle cross-modal incongruence. Prior approaches, from fusion-based methods to in-context learning with Large Vision-Language Models (LVLMs), have made progress but remain limited by three factors: i) cultural blindness (missing symbolic context), ii) boundary ambiguity (satire vs. abuse confusion), and iii) lack of interpretability (opaque model reasoning). We introduce CROSS-ALIGN+, a three-stage framework that systematically addresses these limitations: (1) Stage I mitigates cultural blindness by enriching multimodal representations with structured knowledge from ConceptNet, Wikidata, and Hatebase; (2) Stage II reduces boundary ambiguity through parameter-efficient LoRA adapters that sharpen decision boundaries; and (3) Stage III enhances interpretability by generating cascaded explanations. Extensive experiments on five benchmarks and eight LVLMs demonstrate that CROSS-ALIGN+ consistently outperforms state-of-the-art methods, achieving up to 17% relative F1 improvement while providing interpretable justifications for each decision.
Stanford HAI