Search papers, labs, and topics across Lattice.
This paper addresses the inadequacies of current multimodal safety systems in detecting hidden hateful illusions, where existing classifiers and vision-language models perform poorly. By framing hidden hateful illusion detection as a perceptual retrieval problem, the authors introduce Adaptive View Retrieval, a framework that dynamically selects and calibrates views from a complementary view bank to accurately identify harmful content. The method achieves a remarkable 93.2% balanced accuracy on the HatefulIllusion dataset, significantly outperforming traditional baselines and even human performance on various illusion tasks.
Hidden hateful illusions can be detected with over 93% accuracy using a novel adaptive retrieval approach, revealing the limitations of current moderation systems.
Hateful optical illusions expose a serious gap in current multimodal safety systems. On original-view hateful illusions, previous work shows that six moderation classifiers achieve at most 20.9 to 24.5% accuracy and nine state-of-the-art VLMs remain at or below 10.2% with illusion-aware prompting, leaving most hidden hate undetected. We formulate hidden hateful illusion detection as a perceptual retrieval problem and propose Adaptive View Retrieval. This retrieve-and-calibrate framework assembles a complementary view bank for the image and hidden-message templates, adaptively selects which views to trust, retrieves hidden-message identities, and calibrates whether the recovered evidence is harmful. On HatefulIllusion with a frozen CLIP encoder, Adaptive View Retrieval reaches 93.2% balanced accuracy on the held-out test split. It substantially outperforms original-view baselines and fixed single-transform filters across hate slangs, hate symbols, and visibility levels. The same design also surpasses official fine-tuned CLIP baselines, matches or exceeds human performance on IllusionMNIST, IllusionFashionMNIST, and IllusionAnimals, and outperforms zoom-out preprocessing on HC-Bench under the SemVink protocol. Together, these results show that robust multimodal moderation requires recovering hidden meaning before deciding whether it is harmful.
