Search papers, labs, and topics across Lattice.
The paper introduces GSStream, a volumetric scene streaming system designed to efficiently deliver 3D Gaussian Splatting (3DGS) data. GSStream employs a collaborative viewport prediction module that leverages both collaborative and historical user priors to anticipate future viewing behavior. It also incorporates a deep reinforcement learning (DRL)-based bitrate adaptation module to dynamically adjust the streaming bitrate based on network conditions and user behavior. Experiments demonstrate that GSStream achieves superior visual quality and reduced network usage compared to existing volumetric scene streaming systems.
Stream 3D Gaussian Splatting scenes with higher visual quality and lower bandwidth by predicting user viewpoints and dynamically adapting bitrate using deep reinforcement learning.
Recently, the 3D Gaussian splatting (3DGS) technique for real-time radiance field rendering has revolutionized the field of volumetric scene representation, providing users with an immersive experience. But in return, it also poses a large amount of data volume, which is extremely bandwidth-intensive. Cutting-edge researchers have tried to introduce different approaches and construct multiple variants for 3DGS to obtain a more compact scene representation, but it is still challenging for real-time distribution. In this paper, we propose GSStream, a novel volumetric scene streaming system to support 3DGS data format. Specifically, GSStream integrates a collaborative viewport prediction module to better predict users' future behaviors by learning collaborative priors and historical priors from multiple users and users' viewport sequences and a deep reinforcement learning (DRL)-based bitrate adaptation module to tackle the state and action space variability challenge of the bitrate adaptation problem, achieving efficient volumetric scene delivery. Besides, we first build a user viewport trajectory dataset for volumetric scenes to support the training and streaming simulation. Extensive experiments prove that our proposed GSStream system outperforms existing representative volumetric scene streaming systems in visual quality and network usage. Demo video: https://youtu.be/3WEe8PN8yvA.
