Search papers, labs, and topics across Lattice.
This paper introduces a Bayesian decision theory framework for task-specific next-best-view selection in 3D reconstruction from point clouds, optimizing for downstream task performance rather than uniform uncertainty reduction. They leverage stochastic surface reconstruction to compute a posterior distribution over implicit surfaces, guiding view selection based on its impact on the task. Experiments across semantic classification, segmentation, and PDE-guided physics simulation show improved task performance with fewer views compared to baselines.
Task-aware 3D reconstruction slashes the number of views needed by focusing on the data that actually matters for downstream applications.
We develop a framework for task-specific active next-best-view selection in 3D reconstruction from point clouds, by casting the problem in the language of Bayesian decision theory. Our framework works by (a) placing a prior distribution over the space of implicit surfaces, (b) using recently-developed stochastic surface reconstruction methods to calculate the resulting posterior distribution, then (c) using the posterior distribution to carefully reason about which view to scan next. This enables us to perform camera selection in a manner that is directly optimized for the intended use of the reconstructed data - meaning, we reduce uncertainty only in those regions that make a difference in the task at hand, as opposed to prior approaches that reduce it uniformly across space. We evaluate our method across three distinct downstream tasks: semantic classification, segmentation, and PDE-guided physics simulation. Experimental results demonstrate that our framework achieves superior task performance with fewer views compared to commonly used baselines and prior general uncertainty-reduction techniques.
