Search papers, labs, and topics across Lattice.
This paper introduces VoroTracing, a novel approach to real-time novel view synthesis that leverages differentiable Voronoi ray tracing to achieve high throughput while naturally accommodating non-pinhole effects like distortion and depth of field. By optimizing scene representation, GPU execution, and memory usage, VoroTracing achieves an impressive rendering speed of 623 FPS on an RTX 5090, outperforming previous ray-based methods by 3.2 times and 3D Gaussian Splatting by 2.8 times, all while maintaining competitive reconstruction quality. The method's ability to support complex visual effects without specialized rasterization marks a significant advancement in real-time rendering techniques.
VoroTracing achieves 623 FPS with ray-based rendering, outperforming prior methods while elegantly handling complex visual effects like motion blur and depth of field.
Real-time novel view synthesis is dominated by rasterized explicit primitives. These projection-based pipelines provide high throughput but require specialized extensions for non-pinhole effects such as distortion, rolling shutter, and depth of field. Ray-based rendering expresses these effects naturally but is generally assumed too slow for competitive real-time rendering. We analyze the factors governing throughput in differentiable Voronoi ray tracing and identify traversal length, per-cell work, and memory locality as principal determinants. Guided by this, we introduce VoroTracing, which co-designs the scene representation, optimization, and GPU execution to reduce these costs. Compact octahedral appearance textures reduce memory traffic, while surface-concentrated opacity promotes early termination. The fixed-budget representation is optimized without pruning or densification and rendered with a GPU implementation designed for coherent traversal. On Mip-NeRF 360, VoroTracing renders at 623 FPS on an RTX 5090, providing $3.2\times$ the throughput of the fastest prior ray-based method and $2.8\times$ that of 3D Gaussian Splatting, while maintaining competitive reconstruction quality. Our renderer supports fisheye, rolling-shutter, motion-blur, and depth-of-field effects through ray generation and sampling, requiring no specialized rasterization. These results show that real-time throughput can be achieved with the flexibility of ray-based rendering. We release our source code, see https://research.zenseact.com/publications/vorotracing
