Search papers, labs, and topics across Lattice.
This paper introduces a JEPA-style framework for self-supervised learning on 4D point cloud videos that addresses the limitations of reconstruction-based pretraining by predicting masked spatiotemporal point-tube representations from visible context. By employing Sketched Isotropic Gaussian Regularization, the model stabilizes latent predictions and fosters robust embeddings without relying on explicit reconstruction targets. Experimental results demonstrate significant improvements in downstream tasks such as action and gesture recognition, highlighting the framework's effectiveness in enhancing limited-label learning and cross-dataset transfer.
Masking spatiotemporal regions in 4D point cloud videos leads to more robust representations that outperform traditional reconstruction methods in downstream tasks.
Self-supervised representation learning for 4D point cloud videos is challenging because annotations are costly and reconstruction-based pretraining can overemphasize low-level geometric details. We propose a JEPA-style framework that learns from unlabeled spatiotemporal point clouds through latent point-tube prediction. Instead of reconstructing raw coordinates, the model masks spatiotemporal regions and predicts their target representations from visible context representations in feature space. To stabilize latent prediction, we incorporate Sketched Isotropic Gaussian Regularization, which encourages non-collapsed embeddings without relying on explicit reconstruction targets. This formulation aims to capture both spatial structure and temporal dynamics while keeping the pretraining objective aligned with downstream semantic recognition. Experiments on action and gesture recognition benchmarks show that the learned representations improve downstream fine-tuning, limited-label learning, and cross-dataset transfer. These results suggest that JEPA-style latent prediction is a promising alternative to reconstruction-centered pretraining for 4D point cloud videos.
