Search papers, labs, and topics across Lattice.
The paper introduces GOT-JEPA, a model-predictive pretraining framework that extends Joint-Embedding Predictive Architecture (JEPA) to predict tracking models, enhancing generalization and robustness in generic object tracking. GOT-JEPA trains a student predictor to predict pseudo-tracking models from corrupted frames, using a teacher predictor trained on clean frames, thereby improving resilience to occlusions and distractors. They further propose OccuSolver, which uses a point-centric point tracker to estimate object visibility and capture detailed occlusion patterns, iteratively refining visibility states based on object priors from the tracker.
By predicting tracking models rather than image features, GOT-JEPA unlocks more robust object tracking, even when objects are heavily occluded or the environment is dynamic.
The human visual system tracks objects by integrating current observations with previously observed information, adapting to target and scene changes, and reasoning about occlusion at fine granularity. In contrast, recent generic object trackers are often optimized for training targets, which limits robustness and generalization in unseen scenarios, and their occlusion reasoning remains coarse, lacking detailed modeling of occlusion patterns. To address these limitations in generalization and occlusion perception, we propose GOT-JEPA, a model-predictive pretraining framework that extends JEPA from predicting image features to predicting tracking models. Given identical historical information, a teacher predictor generates pseudo-tracking models from a clean current frame, and a student predictor learns to predict the same pseudo-tracking models from a corrupted version of the current frame. This design provides stable pseudo supervision and explicitly trains the predictor to produce reliable tracking models under occlusions, distractors, and other adverse observations, improving generalization to dynamic environments. Building on GOT-JEPA, we further propose OccuSolver to enhance occlusion perception for object tracking. OccuSolver adapts a point-centric point tracker for object-aware visibility estimation and detailed occlusion-pattern capture. Conditioned on object priors iteratively generated by the tracker, OccuSolver incrementally refines visibility states, strengthens occlusion handling, and produces higher-quality reference labels that progressively improve subsequent model predictions. Extensive evaluations on seven benchmarks show that our method effectively enhances tracker generalization and robustness.
