Search papers, labs, and topics across Lattice.
This paper introduces Evidence Acquisition and Feasibility Gating (EAFG), a novel framework that enhances long-horizon manipulation tasks by integrating Vision-Language Models (VLMs) with Task and Motion Planning (TAMP). EAFG intelligently acquires visual evidence through exploratory subgoals, enabling robots to make informed decisions about task feasibility under conditions of partial observability. Experimental results demonstrate that EAFG significantly improves task completion rates in cooking scenarios by effectively identifying relevant objects and making appropriate halt decisions when necessary.
Robots using EAFG can identify crucial objects before planning, leading to a dramatic increase in successful task completion rates.
Robots executing long-horizon manipulation tasks from natural-language instructions must reason about both semantic task structure and geometric feasibility. However, under partial observability, the availability of goal-relevant objects may be uncertain. In such cases, approaches that combine Vision-Language Models (VLMs) with Task and Motion Planning (TAMP) may generate subgoals that rely on the VLM's prior knowledge without observational support, leading to execution failures or unintended outcomes. We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework that acquires visual evidence through VLM-generated exploratory subgoals and TAMP-based execution. EAFG then applies a feasibility gate to decide whether to proceed with task planning, acquire further evidence, or halt. Our experiments show that, in cooking tasks with ambiguous object use, EAFG improves recipe completion by discovering task-relevant objects before planning. For instructions requiring an absent object, EAFG promotes appropriate halt decisions and reduces repeated attempts to manipulate that object.
