Search papers, labs, and topics across Lattice.
This paper introduces COSTA, a novel approach for annotation-free open-set semantic segmentation of aerial point clouds that addresses the challenges posed by domain shifts during inference. By transitioning from a closed-set, point-wise adaptation to a cluster-centric methodology, COSTA effectively adapts pre-trained models to new target domains without additional training, allowing for the segmentation of categories beyond the original label space. The method achieves a remarkable 70.09% mean Intersection over Union (mIoU) across various benchmarks, demonstrating its efficacy in handling significant domain gaps and mixed semantic shifts.
A single test-time adaptation can enable aerial point cloud segmentation across diverse domains and categories without any additional training, achieving up to 70.09% mIoU.
Semantic segmentation of aerial point cloud is trapped in a generalization crisis under distinct domain shifts. While test-time adaptation offers a privacy-preserving and computationally efficient way to adapt pre-trained models to unlabeled target-domain data during inference, existing methods, bound to closed-set label assumptions and non-scalable point-wise segmentation pipelines, still struggle with semantic shifts. We ask: can we adapt any given pre-trained aerial point cloud segmentation model to a shifted target domain at the inference phase alone, without additional training, while segmenting target-specific categories beyond the source label space on demand? This paper introduces COSTA, which breaks this limitation by shifting from closed-set point-wise adaptation to cluster-centric open-set semantic propagation. Our core discovery is that, once effectively adapted at test time, the rich feature distribution of aerial point clouds can be distilled into a compact set of well-separated semantic centroids that are transferable across label spaces. COSTA leverages this to reformulate open-set semantic segmentation as a cluster-level propagating process: it first bridges the domain gap through proven test-time adaptation, then groups each batch of target-domain points into a small set of semantic clusters based on the similarity distribution in the adapted feature space, and finally propagates high-confidence pseudo labels obtained from an open-vocabulary vision-language model to all points through cluster-level voting. This cluster-centric paradigm enables test-time adaptation of aerial point clouds under significant domain gaps with mixed semantic shifts. With DALES as the source domain, COSTA enables on-demand segmentation across three aerial point cloud benchmarks with distinct domains and heterogeneous category spaces, achieving up to 70.09% mIoU under this new setting.
