Search papers, labs, and topics across Lattice.
The paper introduces ModMap, a novel multiview and multimodal framework for 3D anomaly detection and segmentation that learns to map features across modalities and views. It uses a cross-view training strategy with feature-wise modulation to model view-dependent relationships. Experiments on the SiM3D benchmark show that ModMap achieves state-of-the-art performance, significantly outperforming existing methods.
ModMap's cross-view modulation approach unlocks state-of-the-art 3D anomaly detection, proving that explicitly modeling view-dependent relationships is key to leveraging multiview data.
We present ModMap, a natively multiview and multimodal framework for 3D anomaly detection and segmentation. Unlike existing methods that process views independently, our method draws inspiration from the crossmodal feature mapping paradigm to learn to map features across both modalities and views, while explicitly modelling view-dependent relationships through feature-wise modulation. We introduce a cross-view training strategy that leverages all possible view combinations, enabling effective anomaly scoring through multiview ensembling and aggregation. To process high-resolution 3D data, we train and publicly release a foundational depth encoder tailored to industrial datasets. Experiments on SiM3D, a recent benchmark that introduces the first multiview and multimodal setup for 3D anomaly detection and segmentation, demonstrate that ModMap attains state-of-the-art performance by surpassing previous methods by wide margins.
