Search papers, labs, and topics across Lattice.
This paper introduces Structured Sparse Gaussian Streaming (S$^2$GS), a novel framework for Free-Viewpoint Video (FVV) reconstruction that optimizes performance for Edge-IoT devices by leveraging structure-aware temporal sparsity. By employing a streaming octree for spatial organization and a hierarchical gating mechanism for temporal updates, S$^2$GS significantly reduces per-frame optimization time by 59% and storage costs by 85% compared to existing methods. Extensive evaluations confirm that S$^2$GS achieves high rendering throughput and low energy consumption, making it suitable for resource-constrained environments.
S$^2$GS slashes per-frame optimization time and storage costs while delivering high-quality Free-Viewpoint Videos, making immersive IoT applications feasible on edge devices.
Streaming reconstruction of Free-Viewpoint Videos (FVVs) supports immersive Internet of Things (IoT) services, such as telepresence and digital twin visualization. Existing methods suffer from high per-frame optimization time and large storage footprints, limiting deployment on resource-constrained Edge-IoT devices. To address these challenges, we propose Structured Sparse Gaussian Streaming (S$^2$GS), an FVV reconstruction framework that exploits structure-aware temporal sparsity to selectively update Gaussian residuals, enabling efficient streaming without compromising visual fidelity. In the spatial domain, a streaming octree hierarchically organizes Gaussian residuals, capturing spatial correlations that guide residual updates. In the temporal domain, a structured gating mechanism, comprising hierarchical feature propagation (HFP) and Gumbel-Sigmoid sampling, converts hierarchical dynamic cues into sparse residual update decisions under differentiable optimization. A multi-level discrete scheme is further adopted to provide fine-grained control over residual updates while preserving intricate dynamic details. Extensive experiments across consumer GPUs, industrial edge IoT devices, and a physical telepresence testbed demonstrate that S$^2$GS consistently reduces per-frame optimization time and storage footprint while maintaining competitive visual quality. Compared with QUEEN, S$^2$GS reduces per-frame optimization time by 59% and storage costs by 85% on an RTX 4090 GPU. On the Jetson AGX Orin, S$^2$GS delivers the highest rendering throughput (60+ FPS) and the lowest energy consumption among the evaluated methods, demonstrating its potential for deployment in resource-constrained systems.
