Search papers, labs, and topics across Lattice.
This paper introduces RealityBridge, a novel framework designed to enhance the quality of edited 3D Gaussian Splatting (3DGS) driving videos by addressing the significant Sim-to-Real gap characterized by rendering artifacts and temporal inconsistencies. By employing multimodal controls and a lightweight GateNet for adaptive condition allocation, RealityBridge effectively improves visual realism and stability in long-sequence videos. Experimental results demonstrate that RealityBridge surpasses existing methods in artifact removal, illumination harmonization, and maintaining temporal consistency across driving simulations.
RealityBridge closes the Sim-to-Real gap in 3D driving simulations, achieving superior visual fidelity and temporal stability that existing methods fail to deliver.
Long-tail hazardous scenarios are essential for safety-oriented autonomous driving, yet they are difficult to collect and reproduce at scale. Editable 3D Gaussian Splatting (3DGS) simulation offers a promising alternative by reconstructing real driving scenes and supporting controllable scene editing. However, edited 3DGS-rendered videos still suffer from a significant Sim-to-Real gap, including rendering artifacts, degraded foreground assets, inconsistent illumination, and temporal flickering. Existing restoration and video generation methods are insufficient for this task, as they often fail to jointly repair 3DGS-specific artifacts, improve visual realism, and ensure temporal consistency. To fill this gap, we propose RealityBridge, a structure-preserving and asset-aware Sim-to-Real framework for edited 3DGS driving videos. RealityBridge uses multimodal controls, including rendered videos, foreground masks, edge maps, and semantic masks, together with a lightweight GateNet for adaptive condition allocation across backbone layers. We further construct targeted training data and introduce autoregressive long-video training with reward-guided post-training to improve restoration quality, temporal stability, and hallucination suppression. Extensive experiments on internal and public driving datasets show that RealityBridge outperforms existing methods in artifact removal, illumination harmonization, and long-sequence temporal consistency.
