Search papers, labs, and topics across Lattice.
The paper introduces PackUV, a novel 4D Gaussian representation for volumetric video that maps Gaussian attributes into a sequence of structured, multi-scale UV atlases for compact storage and compatibility with standard video codecs. To fit this representation, they propose PackUV-GS, a temporally consistent fitting method that optimizes Gaussian parameters directly in the UV domain, incorporating a flow-guided Gaussian labeling and video keyframing module. The method's effectiveness is demonstrated on PackUV-2B, a new large-scale multi-view video dataset, showing superior rendering fidelity and scalability compared to existing Gaussian Splatting methods.
Finally, a volumetric video representation that plays nice with existing video codecs, enabling efficient streaming without sacrificing quality.
Volumetric videos offer immersive 4D experiences, but remain difficult to reconstruct, store, and stream at scale. Existing Gaussian Splatting based methods achieve high-quality reconstruction but break down on long sequences, temporal inconsistency, and fail under large motions and disocclusions. Moreover, their outputs are typically incompatible with conventional video coding pipelines, preventing practical applications. We introduce PackUV, a novel 4D Gaussian representation that maps all Gaussian attributes into a sequence of structured, multi-scale UV atlas, enabling compact, image-native storage. To fit this representation from multi-view videos, we propose PackUV-GS, a temporally consistent fitting method that directly optimizes Gaussian parameters in the UV domain. A flow-guided Gaussian labeling and video keyframing module identifies dynamic Gaussians, stabilizes static regions, and preserves temporal coherence even under large motions and disocclusions. The resulting UV atlas format is the first unified volumetric video representation compatible with standard video codecs (e.g., FFV1) without losing quality, enabling efficient streaming within existing multimedia infrastructure. To evaluate long-duration volumetric capture, we present PackUV-2B, the largest multi-view video dataset to date, featuring more than 50 synchronized cameras, substantial motion, and frequent disocclusions across 100 sequences and 2B (billion) frames. Extensive experiments demonstrate that our method surpasses existing baselines in rendering fidelity while scaling to sequences up to 30 minutes with consistent quality.
