Search papers, labs, and topics across Lattice.
NUS
Tsinghua AICASGreat Bay UniversityMacau University of Science and TechnologyUQAug 10, 2026arXiv:2608.09613This paper introduces Uni4R, a novel framework that integrates Optimal Transport and Ordinary Differential Equations to achieve unified continuous-time 4D reconstruction and point tracking. By learning continuous velocity fields as kinematic priors, Uni4R enhances both tasks, allowing for accurate predictions at arbitrary timestamps rather than relying on heuristic methods or integer timestamps. The proposed Flow Matching Guided Decoder (FMGD) and integral-consistency training strategy yield state-of-the-art performance in 4D reconstruction and point tracking, particularly in a new kinematics-aware benchmark.
Achieving state-of-the-art performance in both 4D reconstruction and point tracking, Uni4R leverages continuous velocity fields to model dynamics at any timestamp, breaking free from traditional limitations.
Existing unified 4D reconstruction and point tracking approaches typically rely on heuristic interpolations or just predict at integer timestamps, lacking kinematic coherence and failing to model dynamics at any arbitrary timestamp. In this paper, we propose Uni4R, a framework that unifies these tasks by learning continuous velocity fields through the synergy of Optimal Transport (OT) and Ordinary Differential Equation (ODE). Importantly, this continuous velocity field acts as a kinematic prior that mutually benefits both 4D reconstruction and point tracking. Specifically, we propose the Flow Matching Guided Decoder (FMGD). A global velocity branch first extracts anchor features that capture the global dynamic state of the sequence. Then, FMGD leverages Flow Matching (FM) theory to formulate a probability path defined by OT on the anchor feature manifold, instantiating it as FM-guided velocity features for velocity prediction. This establishes a robust kinematic inductive bias. Meanwhile, a point reconstruction branch provides geometric features. The local velocity prediction module then joint above features and time embeddings, to decode velocities at arbitrary timestamps. To overcome the absence of high-quality ground-truth velocities in fractional frames, we propose an integral-consistency training strategy. This strategy uses an ODE solver to integrate velocities to recover target pointmaps, enabling the model to be supervised end-to-end directly from integer timestamps. Experimental results demonstrate that Uni4R achieves SOTA performance in both 4D reconstruction and point tracking, and achieves SOTA in our new kinematics-aware benchmark at continuous time.
