Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces 3DTV, a feedforward neural network for real-time novel view synthesis from sparse multi-view inputs. It uses a Delaunay-based triplet selection for angular coverage and a pose-aware depth module for efficient feature reprojection and occlusion handling. The method achieves state-of-the-art performance in terms of quality and speed on multi-view video datasets without requiring scene-specific optimization, making it suitable for real-time applications.
Achieve real-time, high-quality novel view synthesis without scene-specific training, unlocking practical applications in AR/VR and telepresence.
Real-time free-viewpoint rendering requires balancing multi-camera redundancy with the latency constraints of interactive applications. We address this challenge by combining lightweight geometry with learning and propose 3DTV, a feedforward network for real-time sparse-view interpolation. A Delaunay-based triplet selection ensures angular coverage for each target view. Building on this, we introduce a pose-aware depth module that estimates a coarse-to-fine depth pyramid, enabling efficient feature reprojection and occlusion-aware blending. Unlike methods that require scene-specific optimization, 3DTV runs feedforward without retraining, making it practical for AR/VR, telepresence, and interactive applications. Our experiments on challenging multi-view video datasets demonstrate that 3DTV consistently achieves a strong balance of quality and efficiency, outperforming recent real-time novel-view baselines. Crucially, 3DTV avoids explicit proxies, enabling robust rendering across diverse scenes. This makes it a practical solution for low-latency multi-view streaming and interactive rendering. Project Page: https://stefanmschulz.github.io/3DTV_webpage/