Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces UniRED, a unified framework for RGB-D video frame interpolation that leverages RGB, depth, and event-based temporal cues to enhance interpolation quality. By addressing the limitations of traditional methods that struggle with geometric consistency and artifacts in RGB-D scenarios, the authors employ a novel bidirectional flow estimation and soft blending technique. Experimental results show that UniRED significantly outperforms existing methods in both photometric fidelity and geometric accuracy, marking a substantial advancement in RGB-D video processing.
Achieving superior photometric fidelity and geometric accuracy, UniRED transforms RGB-D video interpolation by harnessing the power of event-based temporal cues.
High frame-rate RGB-D videos are crucial for a variety of downstream tasks, including motion analysis, dynamic scene understanding, and 3D reconstruction. However, due to hardware and sensing constraints, practical RGB-D cameras are typically limited to low frame rates, making it difficult to capture rapid scene dynamics. Existing video interpolation methods have achieved strong performance on RGB data, but they are not readily applicable to RGB-D scenarios, where they often yield blurry boundaries, visible artifacts, and degraded geometric consistency. Furthermore, motion estimation from only two boundary frames is inherently under-constrained in complex dynamic scenes. Event cameras, by contrast, provide asynchronous measurements with ultra-high temporal resolution, offering dense motion cues. In this paper, we propose a unified multimodal framework for RGB-D video interpolation that jointly exploits RGB appearance, depth geometry, and event-based temporal cues. Specifically, it first extracts and fuses RGB, depth and event cues, then estimates bidirectional flow with motion basis refinement for RGB and Z-axial refinement for depth, and finally synthesizes the target RGB-D frame via bidirectional warping and soft blending. In addition, we construct a new RGB-D-Event dataset to alleviate the scarcity of tri-modal training data. Extensive experiments on a public benchmark and the proposed dataset demonstrate that our method achieves superior photometric fidelity for RGB interpolation and stronger geometric accuracy for depth interpolation than existing approaches.