Search papers, labs, and topics across Lattice.
This paper introduces Misanthrope, a privacy-preserving keypoint detector that is trained through self-distillation to specifically avoid detecting keypoints on people, thereby addressing the vulnerability of local image features to inversion attacks. By mitigating these attacks at the source, Misanthrope enhances privacy in applications like SLAM and SfM without sacrificing performance. The results show that Misanthrope not only matches state-of-the-art image matching performance but also excels in scenarios where people are potential distractors, achieving top performance in 7 out of 9 scenes in the Image Matching Challenge 2021 Phototourism test set.
Misanthrope outperforms traditional feature detectors by preventing the identification of people, thus safeguarding privacy while maintaining high image matching accuracy.
Image matching is a core component of applications such as Simultaneous Localization and Mapping (SLAM), Visual Localization, and Structure from Motion (SfM). However, the local image features central to this task are vulnerable to inversion attacks, which enable adversaries to reconstruct privacy-sensitive scene content from local features. These attacks pose a particular threat in distributed computing scenarios where the pre-computed features leave edge devices to be processed by remote servers. In this work, we introduce Misanthrope, a novel privacy-preserving keypoint detector trained through self-distillation to avoid detecting keypoints on people---a predominant source of privacy-sensitive content in most localization scenarios---thus mitigating inversion attacks at the source rather than through post-hoc obfuscation. We demonstrate how inverted images from traditional feature detection pipelines can be used to detect and re-identify people in the scene, while Misanthrope is able to mitigate these attacks. Furthermore, Misanthrope maintains image matching performance on par with the state of the art and even surpasses it in challenging settings where people act as distractors, such as phototourism and in-the-wild odometry. On the Image Matching Challenge 2021 Phototourism test set, Misanthrope is the top-performing sparse feature extractor in 7 out of 9 scenes. We make our model and its evaluation script available here: https://github.com/fratopa/misanthrope
