Search papers, labs, and topics across Lattice.
This paper introduces the Behavior Prompting Policy (BPP), an innovative visuomotor architecture that allows robots to learn new tasks in real-time from a single human demonstration, termed a behavior prompt. The authors highlight the significance of task diversity in enhancing the prompting capability and present iPhUMI, a handheld interface designed to collect diverse training data. Evaluation methods, including DrawAnything and LIBERO-Gen, demonstrate the effectiveness of BPP in adapting to previously unseen tasks, showcasing a scalable approach to teaching robots without the need for extensive fine-tuning.
Robots can now learn new tasks on-the-fly from just one demonstration, revolutionizing how we teach machines to manipulate their environments.
We study behavior prompting, a paradigm that enables robots to perform new tasks at inference time given a single human demonstration, which we call a behavior prompt. To enable this capability, we present contributions in algorithm, data, and evaluation. For algorithm, we introduce Behavior Prompting Policy (BPP), an in-context visuomotor architecture that translates the behavior prompt and the current observation into robot actions. For data, we identify that task diversity is the primary driver of the prompting capability and introduce iPhUMI, a handheld manipulation interface for collecting diverse training data. For evaluation, we introduce DrawAnything and LIBERO-Gen to evaluate test-time adaptation to unseen drawing and tabletop manipulation tasks. We also demonstrate that iPhUMI serves as a practical interface for specifying behavior prompts at test time, enabling a human to command a robot via a single demonstration to complete known tasks or to define new robot capabilities. Altogether, behavior prompting provides a flexible and scalable way to teach robots new skills without the need for expensive fine-tuning. Our project website is located at https://behavior-prompting.github.io/ .
Stanford HAI