Search papers, labs, and topics across Lattice.
This paper introduces SVP-IL, a novel architecture that decouples semantic reasoning from spatial control in language-conditioned imitation learning for robotic manipulation. By utilizing vision-language foundation models to create zero-shot geometric masks, SVP-IL translates language into explicit Spatial Visual Prompts, which are then integrated into a continuous action generator. Extensive experiments reveal that SVP-IL significantly enhances performance on ambiguous tasks, achieving success rates of 39.5% from just 50 to 100 demonstrations, and 67.8% on standard benchmarks, outperforming existing models.
SVP-IL boosts success rates on ambiguous language tasks by over 60% with minimal training data, revolutionizing data efficiency in robotic manipulation.
While end-to-end Vision-Language-Action (VLA) models show promise in robotic manipulation, their monolithic paradigm inherently couples semantic reasoning and spatial control. This creates a severe alignment bottleneck, limiting precise target disambiguation in data-constrained imitation learning. To overcome this, we propose SVP-IL, a decoupled architecture that explicitly extracts spatial visual grounding from the action generation loop. By leveraging vision-language foundation models, we parse instructions into zero-shot geometric masks, translating language into explicit Spatial Visual Prompts (SVP). These priors are injected into a continuous action generator via a lightweight direct feature-level fusion mechanism. This integration provides explicit and uncorrupted spatial gradient guidance while ensuring highly stable optimization under low-data regimes. Extensive experiments demonstrate that SVP-IL significantly outperforms state-of-the-art VLAs and pure visuomotor baselines. Trained on as few as 50 to 100 demonstrations, SVP-IL improves average success rates on highly ambiguous language-conditioned tasks from 24.0% to 39.5%, achieving 67.8% on standard benchmarks. Real-world robotic experiments further validate its robustness and data efficiency in unstructured physical environments.