Search papers, labs, and topics across Lattice.
ReVo, a loss-resilient volumetric videoconferencing system, addresses the challenges of transmitting volumetric video over unreliable networks by jointly recovering RGB and depth content in real-time. It achieves this by decoupling the video into RGB and depth streams, applying selective forward error correction (FEC) at the network layer to protect critical content, and employing a post-decode neural recovery module to reconstruct corrupted non-critical frames. Experiments using real-world loss traces demonstrate that ReVo significantly improves SSIM for both RGB and depth content and substantially reduces video freezes compared to existing methods.
Volumetric videoconferencing doesn't have to freeze and stutter: ReVo recovers up to 32% of lost RGB data and slashes video freezes by 95% using a cross-layer approach.
Volumetric videoconferencing enables immersive six Degrees of Freedom interactions by jointly transmitting visual appearance and 3D geometry. However, delivering volumetric video over today's networks remains challenging due to high bandwidth demands, strict real-time latency constraints, and frequent packet loss. Packet loss not only degrades visual quality but also corrupts geometric structure, leading to severe artifacts and video freezes that significantly degrade Quality of Experience. Existing solutions either optimize volumetric videos assuming reliable networks or focus on loss recovery for 2D video, and are insufficient for volumetric videoconferencing. In this paper, we present ReVo, a loss-resilient volumetric videoconferencing system that jointly recovers RGB and depth content under packet loss while meeting real-time constraints on desktop-grade hardware. ReVo leverages the insight that effective recovery requires a cross-layer, modality-aware design. It decouples volumetric video into RGB and depth streams, selectively protects critical content using network-layer FEC, and reconstructs corrupted non-critical frames using a post-decode neural recovery module. ReVo is implemented end-to-end over WebRTC and supports both traditional and neural video codecs. Our evaluations using real-world loss traces show that ReVo improves median SSIM by up to 32% (resp. 13%) for RGB (resp. depth) content and reduces video freezes by up to 95.7% compared to existing techniques.
