Search papers, labs, and topics across Lattice.
ReCoSplat is introduced as an autoregressive feed-forward Gaussian Splatting model for online novel view synthesis from sequential observations, addressing the challenge of robust scene reconstruction from unposed inputs. The core innovation is a Render-and-Compare (ReCo) module that renders the current reconstruction from the predicted viewpoint and compares it with the incoming observation, providing a stable conditioning signal that compensates for pose errors. To handle long sequences, ReCoSplat employs a hybrid KV cache compression strategy, achieving state-of-the-art performance across various input settings.
Forget SLAM, ReCoSplat uses a "Render-and-Compare" module to autoregressively refine Gaussian Splatting reconstructions, even from unposed video, achieving SOTA novel view synthesis.
Online novel view synthesis remains challenging, requiring robust scene reconstruction from sequential, often unposed, observations. We present ReCoSplat, an autoregressive feed-forward Gaussian Splatting model supporting posed or unposed inputs, with or without camera intrinsics. While assembling local Gaussians using camera poses scales better than canonical-space prediction, it creates a dilemma during training: using ground-truth poses ensures stability but causes a distribution mismatch when predicted poses are used at inference. To address this, we introduce a Render-and-Compare (ReCo) module. ReCo renders the current reconstruction from the predicted viewpoint and compares it with the incoming observation, providing a stable conditioning signal that compensates for pose errors. To support long sequences, we propose a hybrid KV cache compression strategy combining early-layer truncation with chunk-level selective retention, reducing the KV cache size by over 90% for 100+ frames. ReCoSplat achieves state-of-the-art performance across different input settings on both in- and out-of-distribution benchmarks. Code and pretrained models will be released. Our project page is at https://freemancheng.com/ReCoSplat .
