Search papers, labs, and topics across Lattice.
This paper introduces APPROVE, a multi-modal programming framework that leverages Large Language Models (LLMs) to facilitate robot programming for non-experts by combining natural language input with a block-based interface. By incorporating an explicit user confirmation step, the system enhances transparency and alignment with user intent, allowing users to modify or reject generated programs before execution. The framework also supports the storage of confirmed functions in a library, promoting reusability and building a reliable set of program components over time.
Users can now program robots with confidence, thanks to a system that combines LLMs and visual feedback to ensure intent alignment and reusability.
Programming robots remains challenging for non-experts, as traditional methods require expert knowledge and even block-based interfaces often lack flexibility. Recent work has explored Large Language Models (LLMs) to automatically generate robot programs from natural language, but these systems remain limited by a lack of transparency, missing mechanisms to ensure alignment with user intent, and little support for reuse. We present APPROVE (AI-Powered Programming for Robots with Visual End-User Feedback), an LLM-based multi-modal end-user programming framework that integrates natural language input with a block-based interface and an explicit user confirmation step. Generated programs are visualized using a block-based interface in Blockly, allowing users to confirm, modify, or reject them before execution. Confirmed functions are stored in a library for reuse, gradually building a set of reliable program components. Our approach contributes a human-centered design for LLM-based robot programming that emphasizes user trust, intent alignment, and reusability.
