Search papers, labs, and topics across Lattice.
The paper introduces SR-JEPA, a novel joint-embedding predictive architecture designed for scene-scale point clouds that predicts latent representations of missing entities in 3D scenes. By training on self-contained 3D EMA targets without relying on reconstruction or external features, SR-JEPA achieves a semantic-identity macro accuracy of 43.13% on held-out ARKitScenes objects, significantly outperforming baseline methods. The findings highlight the model's ability to generate context-dependent entity content, which can be effectively utilized in downstream computations involving metric geometry.
SR-JEPA reveals that a predictive pathway can infer missing entity representations in 3D scenes with remarkable accuracy, transforming how we approach latent state learning.
Joint-embedding predictive architectures learn by predicting latent representations of missing observations, yet many masked JEPAs are evaluated primarily through the encoders they produce. We ask what a trained predictive pathway itself infers when an entire entity is absent from a native 3D scene. We introduce SR-JEPA, a point-native JEPA for scene-scale point clouds whose original frozen predictive pathway can be queried at a supplied location. At evaluation, every point of one object is removed before encoding and replaced by the same shape-free 32-point query at its centroid. Training uses only self-contained 3D EMA targets: no reconstruction, semantic labels, language, or lifted 2D features. On 5,953 held-out ARKitScenes objects, the imputed latent reaches 43.13% semantic-identity macro accuracy, 22.18 points above the strongest floor. Randomizing the prediction path removes 9.78 points, while substituting matched donor context removes 21.98 points. On 8,570 Sr3D support pairs, the full latent reaches 41.15 AP; identity decoded from the missing-object latent, combined with anchor identity and geometry, reaches 39.37 AP, leaving an unresolved 1.78-point residual. These results reveal a queryable, compositional 3D predictive state: the model completes context-dependent entity content, which downstream computation combines with metric geometry.
