Search papers, labs, and topics across Lattice.
This paper introduces Voice in Head (ViH), a novel framework using LLMs as Actor and Critic components within a reinforcement learning loop to improve robot navigation and interaction. The system uses GPT and Gemini powered LLMs, Azure AI Search for semantic understanding, and RLHF for safety. ViH achieves a 94.54% success rate in navigation tasks, outperforming existing benchmarks, while also demonstrating modularity and scalability across different environments.
Robots can now navigate complex environments with 94% accuracy by combining the reasoning power of GPT/Gemini with human feedback, suggesting a practical path to more robust and adaptable autonomous systems.
This work presents a novel Voice in Head (ViH) framework, that integrates Large Language Models (LLMs) and the power of semantic understanding to enhance robotic navigation and interaction within complex environments. Our system strategically combines GPT and Gemini powered LLMs as Actor and Critic components within a reinforcement learning (RL) loop for continuous learning and adaptation. ViH employs a sophisticated semantic search mechanism powered by Azure AI Search, allowing users to interact with the system through natural language queries. To ensure safety and address potential LLM limitations, the system incorporates a Reinforcement Learning with Human Feedback (RLHF) component, triggered only when necessary. This hybrid approach delivers impressive results, achieving success rates of up to 94.54%, surpassing established benchmarks. Most importantly, the ViH framework offers a modular and scalable architecture. By simply modifying the environment, the system demonstrates the potential to adapt to diverse application domains. This research provides a significant advancement in the field of cognitive robotics, paving the way for intelligent autonomous systems capable of sophisticated reasoning and decision-making in real-world scenarios bringing us one step closer to achieving Artificial General Intelligence.
