Search papers, labs, and topics across Lattice.
HorizonStream addresses the limitations of existing online 3D reconstruction methods, which struggle with drift and instability due to their inability to handle the temporally heterogeneous nature of streaming geometry. The paper introduces a long-horizon Transformer architecture that factorizes the evidence influence kernel into long-range temporal and short-range spatial components, using Geometric Linear Attention with learned decay rates and Geometric Local Attention with Spatiotemporal RoPE, respectively. This approach enables stable, constant-memory, and linear-time reconstruction on sequences exceeding 10,000 frames, achieving state-of-the-art performance.
Streaming 3D reconstruction over thousands of frames is now possible with constant memory and linear time, thanks to a novel attention mechanism that learns to propagate geometric evidence across multiple timescales.
Online 3D reconstruction requires estimating camera pose and scene geometry under strict causal and bounded-memory constraints. Existing methods often suffer from drift, jitter, or collapse on long sequences. We trace these failures to a fundamental mismatch. Streaming geometry is inherently temporally heterogeneous, with evidence ranging from short-lived correspondences to persistent global scale. However, current architectures impose uniform and pathological influence patterns. For example, sliding windows enforce hard cutoffs, while ungated recurrence and causal attention cause cache saturation and spike-like attention sinks. To resolve this, we formalize geometric propagation as an evidence influence kernel and propose HorizonStream, a long-horizon Transformer that explicitly factorizes this kernel. For the long-range temporal factor, Geometric Linear Attention learns channel-wise decay rates to enable bounded, multi-timescale propagation of geometric evidence. For the short-range spatial factor, Geometric Local Attention with Spatiotemporal RoPE performs reliable 3D matching while suppressing attention sinks. Finally, Metric Readout Tokens recover stable scale and rigid pose directly from the persistent geometric state. Extensive experiments show that HorizonStream, trained on only 48-frame clips, generalizes stably to sequences exceeding 10,000\ frames with constant memory and linear time, achieving state-of-the-art streaming 3D reconstruction performance. Project Page: https://3dagentworld.github.io/horizonstream/
