Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces AQ3D, an Adaptive Query Transformer designed for 3D instance segmentation that dynamically adjusts the number of queries based on the scene's characteristics. By utilizing a fixed ratio of queries to superpoints and employing 3D RoPE for positional encoding, AQ3D effectively addresses the limitations of traditional fixed-query methods, which struggle with varying scene sizes and object counts. Experimental results demonstrate that AQ3D achieves state-of-the-art performance on multiple datasets, including ScanNetV2 and ScanNet++, without relying on additional data augmentation.
AQ3D achieves unprecedented adaptability in 3D instance segmentation, outperforming existing methods by dynamically tailoring query counts to scene complexity.
Transformer-based decoders for 3D instance segmentation typically commit to a fixed number of queries and positional modeling calibrated on the training distribution rather than on the scene at hand. Indoor scans vary widely in spatial extent and object count, so a fixed query set over-initializes small scenes and under-initializes large ones, while learned absolute and relative encodings are bound to the training scenes' extents and can saturate. We present AQ3D, which is designed to handle scenes of various sizes during training and inference. Queries are instantiated at a fixed ratio of the scene's superpoints, forming an overcomplete set whose background rejection is entirely left to the decoder. Positional information is encoded using 3D RoPE over quantized metric coordinates, replacing learned bounded lookup tables of prior decoders. Further, we improve the decoder itself by using attribution-based superpoint pooling, a mask refinement branch, and a cosine classifier for background rejection. Experiments show our method sets a new state-of-the-art on validation and hidden test splits across the datasets ScanNetV2, ScanNet200, and ScanNet++V2 among decoder methods trained without additional data augmentation. Code is available at \href{https://github.com/kenomo/aq3d}{github.com/kenomo/aq3d}.