Search papers, labs, and topics across Lattice.
This paper introduces a modality-agnostic prompt learning framework for camouflaged object detection (COD) that leverages the Segment Anything Model (SAM). The approach distills task-relevant cues from multi-modal inputs (RGB-Depth, RGB-Thermal, RGB-Polarization) into unified prompts for SAM, enabling parameter-efficient adaptation. Results demonstrate improved performance and generalization across different multi-modal COD benchmarks, validated through extensive experimentation.
Forget modality-specific architectures: this work achieves state-of-the-art camouflaged object detection by learning modality-agnostic prompts for SAM, unlocking efficient adaptation to new modalities.
Camouflaged Object Detection (COD) aims to segment objects that blend seamlessly into complex backgrounds, with growing interest in exploiting additional visual modalities to enhance robustness through complementary information. However, most existing approaches generally rely on modality-specific architectures or customized fusion strategies, which limit scalability and cross-modal generalization. To address this, we propose a novel framework that generates modality-agnostic multi-modal prompts for the Segment Anything Model (SAM), enabling parameter-efficient adaptation to arbitrary auxiliary modalities and significantly improving overall performance on COD tasks. Specifically, we model multi-modal learning through interactions between a data-driven content domain and a knowledge-driven prompt domain, distilling task-relevant cues into unified prompts for SAM decoding. We further introduce a lightweight Mask Refine Module to calibrate coarse predictions by incorporating fine-grained prompt cues, leading to more accurate camouflaged object boundaries. Extensive experiments on RGB-Depth, RGB-Thermal, and RGB-Polarization benchmarks validate the effectiveness and generalization of our modality-agnostic framework.
