Search papers, labs, and topics across Lattice.
This paper introduces On-the-Fly3R, a novel framework for online 3D reconstruction that addresses the challenges of large-scale UAV mapping by enabling reconstruction from unordered image streams. By employing a retrieval-guided dynamic subset construction method and a validation-rejection-retry mechanism, the framework ensures global consistency and effectively handles misaligned images. Evaluations demonstrate that On-the-Fly3R significantly enhances the accuracy of various 3R models when processing over 5,000 images in expansive UAV scenarios, outperforming state-of-the-art streaming methods.
On-the-Fly3R achieves robust online 3D reconstruction from unordered UAV images, significantly improving accuracy in large-scale mapping scenarios.
While feed-forward 3D reconstruction (3R) offers efficient end-to-end modeling, its application in large-scale UAV mapping is hindered by the prohibitive memory cost of Transformer attention. Current scalable streaming 3R methods assume temporally and spatially continuous inputs, rendering them ineffective for the weakly ordered or unordered image streams common in cross-strip UAV operations. To address this, we propose On-the-Fly3R, a training-free, progressive online 3D reconstruction framework for large-scale UAV images that upgrades various 3R backbones for large-scale UAV scenarios. Our method enables reconstruction from unordered inputs via retrieval-guided dynamic subset construction, which adaptively selects spatially relevant images. To further improve the robustness, a validation-rejection-retry mechanism is designed to guarantee global consistency, performing a pre-integration consistency check and automatically rejecting misaligned images and retrying with alternative subset. Finally, inspired by VSLAM, pose graph optimization based on the retrieval loop closure is employed to mitigate camera drift. Evaluations on several UAV benchmarks show that our On-the-Fly3R successfully scales various 3R models to over 5,000 images across square-kilometer UAV scenes, delivering substantially superior accuracy compared to several SOTA streaming 3R methods. Code is available at https://github.com/Sh1nZzz/On_the_Fly3R
