Search papers, labs, and topics across Lattice.
This paper introduces Lift3D-VLA, a unified Vision-Language-Action (VLA) framework that enhances robotic manipulation by integrating explicit 3D point cloud reasoning with temporally coherent action generation. The method employs an improved 2D model-lifting strategy to align 3D points with pretrained 2D embeddings, minimizing spatial information loss, and utilizes Geometry-Centric Masked Autoencoding (GC-MAE) for self-supervised learning of both current and future geometric states. Experimental results demonstrate that Lift3D-VLA achieves significant performance improvements, with up to 11.1% higher success rates in simulated and real-world tasks compared to existing VLA methods, showcasing its superior generalization capabilities.
Achieving over 10% improvement in manipulation success rates, Lift3D-VLA redefines the integration of 3D geometry and action generation in robotic systems.
Recently, Vision-Language-Action (VLA) models have demonstrated strong generalization across diverse tasks. However, effective robotic manipulation in physical environments fundamentally requires geometric understanding and spatial reasoning. While some VLA approaches attempt to incorporate 3D information, they are constrained by limited data availability and geometric information loss in current 3D encoding pipelines, and fail to jointly capture 3D geometry and temporally structured actions in dynamic environments. To address these limitations, we introduce Lift3D-VLA, a unified VLA framework that equips models with explicit 3D point cloud reasoning and enables temporally coherent action generation. First, building upon our previous work Lift3D, an enhanced 2D model-lifting strategy is proposed to geometrically align 3D points with pretrained 2D positional embeddings. This design enables direct point-cloud encoding within the VLA vision encoder while minimizing spatial information loss. Based on explicit 3D inputs, we propose Geometry-Centric Masked Autoencoding (GC-MAE), a dual-objective self-supervised framework that reconstructs the current point cloud while predicting its future geometric evolution. This formulation allows the 2D vision encoder to internalize both 3D structure and physical dynamics. To fully exploit 3D representations, we further design layer-wise temporal action modeling, which leverages multiple layers of the LLM to collaboratively predict action chunks, enabling temporally consistent predictions. Across 22 simulated tasks and 8 real-world manipulation tasks, Lift3D-VLA achieves 10.8% and 11.1% higher mean success rates on MetaWorld and RLBench than the best-performing prior VLA methods, and outperforms the strongest real-world baseline by 4 percentage points, while exhibiting stronger generalization to out-of-distribution perturbations.
