Search papers, labs, and topics across Lattice.
This paper introduces a decentralized framework leveraging LLMs to coordinate acoustic robots for contactless object manipulation based on natural language commands. The system uses Whisper for speech recognition, LLMs for semantic parsing into a structured JSON task representation, and distributed scheduling to coordinate robots. Experiments with two TurtleBot3-based acoustic robots demonstrate high task success rates across sequential (96%), parallel (86%), and synchronized (70%) manipulation scenarios.
LLMs can now orchestrate fleets of acoustic robots to perform complex contactless manipulation tasks from simple spoken commands.
Natural language interfaces can simplify interaction with multi-robot systems, especially when non-expert users need to issue high-level commands. Acoustic manipulation using ultrasonic phased arrays also enables contactless object handling for applications such as healthcare, laboratory automation, and precision transport. However, combining large language models (LLMs) with distributed acoustic mobile robots remains underexplored. This paper presents a decentralized framework for natural language-driven coordination of acoustic robots for contactless object manipulation. The system converts spoken instructions into executable multi-robot task plans using Whisper-based speech recognition, LLM-based semantic parsing, structured JSON task representation, and distributed scheduling. The JSON schema encodes robot assignments, temporal dependencies, spatial constraints, and synchronization requirements for sequential, parallel, and synchronized execution. The system is implemented on two TurtleBot3-based acoustic robots, each equipped with an ultrasonic phased array for contactless object transport. Experiments were conducted in three scenarios: sequential execution, parallel multi-robot transport, and synchronized cooperative manipulation. The system achieved task success rates of 96 percent for sequential tasks, 86 percent for parallel execution, and 70 percent for synchronized collaborative transport. These results show that natural language commands can be transformed into distributed robot actions for contactless manipulation, highlighting the potential of LLM-driven automation for human-robot interaction in distributed robotic systems.
