Search papers, labs, and topics across Lattice.
This paper introduces PUF, a novel framework for online 3D scene graph generation that incorporates uncertainty into the fusion process of 2D observations into a global 3D graph. By reformulating node association as a probabilistic likelihood rather than a deterministic binary decision, PUF effectively addresses previously overlooked sources of uncertainty in observation, 2D models, and 3D representations. Experimental results on the 3DSSG and ReplicaSSG benchmarks demonstrate that PUF significantly outperforms existing methods while achieving real-time processing speeds, highlighting the effectiveness of uncertainty-aware fusion in enhancing online 3D scene understanding.
Uncertainty-aware fusion can dramatically improve online 3D scene graph generation, outperforming traditional methods while maintaining real-time performance.
Online 3D scene graph generation builds a persistent, structured representation of a scene by incrementally fusing 2D observations into a global 3D graph. Existing online methods treat this fusion as a fully deterministic pipeline, where we identify three sources of uncertainty that are overlooked: observation, 2D model, and 3D representation. We propose PUF: a Plug-and-play, Uncertainty-aware, and training-free Fusion framework. Scene graph node association is reformulated as a probabilistic likelihood over semantic and spatial factors, replacing binary accept/reject gates. Dirichlet evidence accumulation distributes class and relationship evidence across plausible candidates proportional to association likelihood. An optional class-conditional prior completes edges for sparsely or never co-observed object pairs. We instantiate PUF with both a 3D Gaussian and a 3D voxel backend and observe consistent improvements, demonstrating its ability to generalize across different representations. Experiments on the 3DSSG and ReplicaSSG benchmarks show that our method substantially outperforms existing approaches while maintaining real-time latency. These results establish uncertainty-aware fusion as a principled and effective paradigm for online 3D scene understanding. The source code is publicly available at https://github.com/yyyyangyi/PUF.
