Search papers, labs, and topics across Lattice.
This paper introduces NoPA, a novel approach to 3D scene graph generation that replaces traditional point-cloud representations with non-parametric distributions for each object, thereby preserving 3D geometric detail while enabling real-time performance. By employing a tailored merging strategy based on maximum mean discrepancy, NoPA effectively addresses the inaccuracies in object merging that arise from approximating objects with single 3D Gaussians. Experimental results demonstrate that NoPA significantly outperforms existing methods in terms of accuracy without compromising inference speed, marking a substantial advancement in online scene understanding.
NoPA retains rich 3D geometric information while achieving real-time scene graph generation, outperforming traditional methods that sacrifice detail for speed.
Classic 3D scene graph generation approaches fail to work in real-time due to the heavy computational cost of environment mapping and the need to generate intermediate point-cloud representations. To alleviate this issue, a recent work eschews point clouds in favor of a lightweight Gaussian distribution for each object. This approximation drastically speeds up inference and enables real-time 3D scene graph generation. However, the representation has two key weaknesses. \textbf{1)} Each object is approximated by a single 3D Gaussian, which causes a severe loss of 3D geometric detail. \textbf{2)} The discrepancy between this approximation and the true object geometry exacerbates the inaccurate merging of object candidates during online inference. To address these issues, we propose \textbf{NoPA}, which represents each object as a separate non-parametric distribution. This formulation retains 3D geometric information while preserving real-time inference of the parametric Gaussian formulation. To build upon our novel object representation, we propose a tailored merging strategy to recover coherent object instances. Specifically, we leverage maximum mean discrepancy on kernel density estimates to enable robust merging of object candidates during online exploration while minimizing added computational complexity. The key is to maintain a fixed particle set per object. Furthermore, to rectify the relation loss caused by misclassified objects, NoPA propagates relationships between objects with high affinity. Experiments show that NoPA substantially outperforms current methods without sacrificing real-time inference speed.
