Search papers, labs, and topics across Lattice.
BRDFusion integrates physically-based rendering with generative models to enhance inverse rendering of urban scenes from video inputs. This hybrid approach addresses the limitations of traditional methods by ensuring consistent scene property recovery while minimizing artifacts through generative denoising. The framework not only produces high-quality videos but also enables advanced functionalities like novel-view relighting and dynamic object manipulation, outperforming existing techniques in both real and synthetic environments.
By merging physical modeling with generative priors, BRDFusion achieves unprecedented control and quality in urban scene rendering, setting a new standard for inverse rendering applications.
Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/
