Search papers, labs, and topics across Lattice.
The authors introduce a large-scale, dynamic dataset called Generative World Renderer (GWR) comprising 4M frames of synchronized RGB and G-buffer data extracted from AAA games using a novel dual-screen capture method. This dataset is designed to address the domain gap in generative inverse and forward rendering by providing visually complex, temporally coherent data with diverse scenes and effects. Experiments demonstrate that models fine-tuned on GWR exhibit improved cross-dataset generalization and controllable generation, validated through a novel VLM-based evaluation protocol that correlates well with human judgment.
AAA game engines are now a viable source of high-quality, temporally coherent training data for generative inverse and forward rendering, enabling more realistic and controllable video generation.
Scaling generative inverse and forward rendering to real-world scenarios is bottlenecked by the limited realism and temporal coherence of existing synthetic datasets. To bridge this persistent domain gap, we introduce a large-scale, dynamic dataset curated from visually complex AAA games. Using a novel dual-screen stitched capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, and environments, including adverse weather and motion-blur variants. This dataset uniquely advances bidirectional rendering: enabling robust in-the-wild geometry and material decomposition, and facilitating high-fidelity G-buffer-guided video generation. Furthermore, to evaluate the real-world performance of inverse rendering without ground truth, we propose a novel VLM-based assessment protocol measuring semantic, spatial, and temporal consistency. Experiments demonstrate that inverse renderers fine-tuned on our data achieve superior cross-dataset generalization and controllable generation, while our VLM evaluation strongly correlates with human judgment. Combined with our toolkit, our forward renderer enables users to edit styles of AAA games from G-buffers using text prompts.
