Search papers, labs, and topics across Lattice.
HDR-NSFF introduces a novel framework for reconstructing dynamic HDR radiance fields from alternating-exposure monocular videos by representing the scene as a continuous 4D spatio-temporal function. The method integrates HDR radiance, 3D scene flow, geometry, and tone-mapping within a unified end-to-end pipeline, leveraging both neural radiance fields and 4D Gaussian Splatting. Key to its robustness are exposure-invariant motion estimation using DINO features and a generative prior to regularize the reconstruction, especially in areas with limited observation or saturation.
Achieve ghost-free, temporally coherent HDR video from a monocular camera by modeling dynamic scenes as 4D radiance fields, even with challenging exposure variations.
Radiance of real-world scenes typically spans a much wider dynamic range than what standard cameras can capture. While conventional HDR methods merge alternating-exposure frames, these approaches are inherently constrained to 2D pixel-level alignment, often leading to ghosting artifacts and temporal inconsistency in dynamic scenes. To address these limitations, we present HDR-NSFF, a paradigm shift from 2D-based merging to 4D spatio-temporal modeling. Our framework reconstructs dynamic HDR radiance fields from alternating-exposure monocular videos by representing the scene as a continuous function of space and time, and is compatible with both neural radiance field and 4D Gaussian Splatting (4DGS) based dynamic representations. This unified end-to-end pipeline explicitly models HDR radiance, 3D scene flow, geometry, and tone-mapping, ensuring physical plausibility and global coherence. We further enhance robustness by (i) extending semantic-based optical flow with DINO features to achieve exposure-invariant motion estimation, and (ii) incorporating a generative prior as a regularizer to compensate for limited observation in monocular captures and saturation-induced information loss. To evaluate HDR space-time view synthesis, we present the first real-world HDR-GoPro dataset specifically designed for dynamic HDR scenes. Experiments demonstrate that HDR-NSFF recovers fine radiance details and coherent dynamics even under challenging exposure variations, thereby achieving state-of-the-art performance in novel space-time view synthesis. Project page: https://shin-dong-yeon.github.io/HDR-NSFF/
