Search papers, labs, and topics across Lattice.
This paper introduces NegROI, a transformer-based framework that enhances interactive 3D segmentation by refining object masks around user clicks using multi-resolution techniques and scene-conditioned negative prompts. It addresses the challenges of coarse voxel resolution and hard false positives, particularly in datasets with varying density and scale, by employing uncertainty-driven selective refinement and boundary-aware hard negative mining. Experimental results on benchmark datasets show that NegROI significantly improves click efficiency and reduces false positives, outperforming existing state-of-the-art methods in cross-dataset robustness.
NegROI achieves superior 3D segmentation with fewer clicks by intelligently refining only the most ambiguous regions while effectively managing background confusion.
Interactive 3D segmentation aims to extract object masks in point clouds with minimal user clicks. Despite recent progress, most existing approaches still struggle with (i) coarse voxel resolution that blurs fine boundaries under limited clicks and (ii) hard false positives caused by confusing background structures. These issues are exacerbated by density and scale shifts across datasets (e.g., dense RGB-D reconstructions vs. sparse LiDAR scans), where fixed refinement heuristics and purely click-driven decoding generalize poorly. To address them, we propose NegROI -- a novel transformer-based interactive framework that couples click-centric multi-resolution refinement with scene-conditioned negative prompts. Given a coarse voxel prediction, it refines only a local Region Of Interest (ROI) around the current click on a finer grid and fuses refined logits back to the coarse mask. To improve robustness and efficiency, we introduce uncertainty-driven selective refinement that prioritizes ambiguous regions. Meanwhile, we model hard background patterns via a set of scene-conditioned negative prompts obtained by cross-attention over scene tokens. We further stabilize these prompts with a diversity regularizer. Finally, we propose boundary-aware hard negative mining to supervise negative-prompt attention toward boundary-proximal, high-confidence false positives. Our experiments on common benchmark datasets (i.e., ScanNet, S3DIS, and KITTI) demonstrate improved click efficiency and reduced false positives, with stronger cross-dataset robustness than the state-of-the-art baselines.
