Search papers, labs, and topics across Lattice.
This study investigates the presence of intuitive-physics knowledge in pretrained video foundation models by employing frozen-feature probing across various model families and layers. The analysis reveals that V-JEPA outperforms other models, particularly in tasks involving temporal dynamics, while VideoMAE shows competitive performance and LTX-Video captures weaker signals. Notably, the findings indicate that intuitive-physics information is least accessible in early layers and significantly influenced by the pretraining approach and the order of input frames.
Intuitive-physics knowledge in video models is not only present but also varies dramatically with model architecture and layer depth, revealing critical insights into representation learning.
We study whether pretrained video foundation models encode intuitive-physics information in their frozen representations, and how this information varies across model families, layers, and probe types. Using frozen-feature probing on IntPhys2 and Minimal Video Pairs (MVP), we compare predictive joint-embedding models (V-JEPA), masked reconstruction models (VideoMAE), and a diffusion-based video generator (LTX-Video). V-JEPA achieves the strongest overall results across benchmarks, especially with probes that model temporal dynamics, while VideoMAE remains competitive and LTX-Video recovers weaker but non-trivial signal. Layerwise analyses show that physics-relevant information is weakest in early layers and becomes most accessible at intermediate-to-late depth, and temporal controls show that disrupting frame order substantially reduces performance, especially on MVP. Together, these results suggest that intuitive-physics knowledge emerges reliably in pretrained video representations, but its accessibility depends strongly on pretraining paradigm, representational depth, and readout mechanism.
