Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Point4D enables feed-forward, dense 3D point trajectory estimation across videos spanning hundreds of frames by decoupling trajectory prediction from image-plane visibility constraints. The framework overcomes the severe context window limitations of prior feed-forward 4D models by directly passing predicted 3D endpoints as queries into subsequent video chunks without reprojection or explicit feature matching. Benefiting from this query chaining and dynamic visual descriptor updates from visible frames, the approach establishes state-of-the-art tracking accuracy on benchmarks exceeding 200 frames.
Dense 4D reconstruction no longer collapses after a few dozen frames: directly re-querying 3D trajectory endpoints across video chunks breaks the long-standing temporal horizon bottleneck in feed-forward dynamic scene reconstruction.
We introduce Point4D, a feed-forward model for 4D reconstruction of long-range video sequences. Point4D is able to reliably infer dense per-point 3D trajectories across multi-hundred-frame videos, unlike existing 4D methods that are limited to short input windows of at most a few dozen frames. A key innovation that enables this is our flexible 3D query-based motion decoder that decouples trajectory prediction from image-plane visibility. The predicted 3D endpoints are then directly re-queried in the next chunk without re-projection or matching. Furthermore, we show that extracting and reusing a visual descriptor from an arbitrary frame where the point is visible leads to better performance than relying solely on the source patch. Overall, Point4D achieves state-of-the-art performance across diverse long-video tracking benchmarks spanning over 200 frames and largely outperforms previous feed-forward 4D method. Project page: https://point-4d.github.io
CMU ML