Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
SHRIMP is a novel system that enables users to generate and iteratively refine robot task plans from natural language inputs, addressing the challenges of semantic ambiguity and lack of transparency in generative models. By allowing users to validate their plans in a simulated environment before execution, SHRIMP enhances user control and understanding of robot actions. A user study with 35 participants demonstrated significant improvements in perceived control and transparency when using SHRIMP for planning tabletop kitchen tasks.
Users can now iteratively refine robot task plans in natural language, significantly boosting their control and understanding of robotic actions.
As collaborative robots have entered domains such as manufacturing, agriculture, and healthcare, programming or adapting robot behavior typically requires robotic expertise that most end users lack. Natural language lowers this barrier. Recent advancements in large language models (LLMs) have made it feasible to translate natural language into robot task plans. However, language-based task specification suffers from semantic ambiguity, and generative models lack transparency for how language instructions become robot actions, making it difficult for users to validate the plan before execution. To address these issues, we introduce SHRIMP, a system that allows users to automatically generate a hierarchical robot primitive plan using natural language and iteratively revise their plan through re-prompting and explicit correction. At each revision, SHRIMP allows users to validate their plan in simulation, and once satisfied, execute it on the physical robot. Through a user study involving participants planning tabletop kitchen tasks (n=35), we validate that SHRIMP improves perceived control and enhances robot transparency. System videos and source code are available at https://wisc-hci.github.io/SHRIMP.