Search papers, labs, and topics across Lattice.
HyperGS introduces a feedforward, optimization-free approach to Gaussian video representation that predicts Gaussian representations directly from any video in a single forward pass. This method significantly accelerates encoding and decoding processes, achieving speeds of $10^4$–$10^5\times$ faster than traditional per-video optimization while maintaining high reconstruction quality. Notably, HyperGS demonstrates improved PSNR by 2.9–3.1 dB over existing video encoders, showcasing its ability to generalize effectively to out-of-distribution videos at higher resolutions without re-encoding.
Achieving encoding speeds up to 100,000 times faster than traditional methods, HyperGS redefines the efficiency of video representation while enhancing quality and generalization.
Gaussian Splatting has emerged as an effective representation for video, but existing methods rely on per-video optimization. This leads to slow encoding and limits generalization across videos. To amortize this optimization, we propose HyperGS, a feedforward, optimization-free approach that directly predicts Gaussian representations from any video in a single forward pass, speeding up encoding and decoding by orders of magnitude while generalizing to out-of-distribution videos at higher resolutions. In HyperGS, we design a factorized spatiotemporal Transformer to extract tokens from video, and a learnable query-based Transformer to obtain 8-parameter Gaussian representations for each video frame. We find that naively predicting Gaussians across diverse videos induces a needle-like degeneration that collapses training, and address this with a rank-based geometric regularizer whose strength adapts dynamically to stabilize optimization. HyperGS achieves encoding at $10^4$--$10^5\times$ the speed of per-video Gaussian optimization at matched reconstruction quality while generalizing zero-shot to $720p$ video, enabling higher-resolution rendering without re-encoding. HyperGS improves PSNR by +2.9--3.1 dB over the prior video encoders on K400, SSv2, and UCF101 at a smaller video representation size. By predicting explicit 2D Gaussians in a single forward pass, HyperGS combines the fast, flexible rendering of Gaussian Splatting with the speed and generalization of feedforward prediction, advancing Gaussians as a practical direction for fast and generalizable video representation.
