Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors introduce a large-scale, dynamic dataset of 4M frames extracted from AAA games, featuring synchronized RGB and G-buffer channels, to address the domain gap in generative inverse and forward rendering. They propose a novel VLM-based evaluation protocol for inverse rendering, measuring semantic, spatial, and temporal consistency in the absence of ground truth. Experiments show that fine-tuning inverse renderers on this dataset improves cross-dataset generalization and controllable generation, with the VLM evaluation aligning well with human perception.
Training generative inverse renderers on AAA game footage closes the realism gap, enabling superior generalization and controllable generation compared to models trained on existing synthetic datasets.
Scaling generative inverse and forward rendering to real-world scenarios is bottlenecked by the limited realism and temporal coherence of existing synthetic datasets. To bridge this persistent domain gap, we introduce a large-scale, dynamic dataset curated from visually complex AAA games. Using a novel dual-screen stitched capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, and environments, including adverse weather and motion-blur variants. This dataset uniquely advances bidirectional rendering: enabling robust in-the-wild geometry and material decomposition, and facilitating high-fidelity G-buffer-guided video generation. Furthermore, to evaluate the real-world performance of inverse rendering without ground truth, we propose a novel VLM-based assessment protocol measuring semantic, spatial, and temporal consistency. Experiments demonstrate that inverse renderers fine-tuned on our data achieve superior cross-dataset generalization and controllable generation, while our VLM evaluation strongly correlates with human judgment. Combined with our toolkit, our forward renderer enables users to edit styles of AAA games from G-buffers using text prompts.