Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces GRASP, a novel framework that utilizes a pretrained Vision-Language Model to translate natural-language prompts into neuro-symbolic goal states for robotic manipulation. By employing a bounding-box detection pipeline, GRASP allows robots to interpret abstract spatial concepts and execute tasks in real time without the need for extensive training or fine-tuning. The framework demonstrates a 73.3% success rate across 90 real-robot trials, showcasing its effectiveness in open-vocabulary tabletop manipulation tasks.
Robots can now understand and execute complex tasks based on natural language without any task-specific training, achieving a 73.3% success rate in real-world trials.
For robotics to be effectively integrated into household or industrial environments, machines must adapt to natural-language prompts in real time. Although Vision-Language Models (VLMs) have enabled zero-shot generalization in robot task and motion planning (TAMP), current state-of-the-art approaches often remain computationally"heavyweight"or require extensive training on thousands of demonstrations. We present GRASP (Grounded Reasoning and Symbolic Planning), a framework designed as a step toward open-vocabulary tabletop manipulation. Our approach leverages a pretrained VLM to translate natural-language queries into neuro-symbolic goal states, grounded in the physical world via a bounding-box detection pipeline. Unlike methods that rely on fixed color lists or hard-coded coordinates, GRASP enables robots to interpret abstract spatial concepts such as"top shelf"and execute tasks without additional fine-tuning. We achieve 73.3% overall success across 90 real-robot trials at three difficulty levels, requiring no task-specific training.