Search papers, labs, and topics across Lattice.
This paper introduces a framework for coaching reinforcement learning agents to exhibit specific styles while performing complex tasks, leveraging universal value function approximators (UVFAs) alongside tailored training scenarios and data augmentation. The approach was validated across diverse domains, including AAA video games and humanoid robotics, demonstrating that agents can maintain coherence with user-defined styles without compromising task performance. The key finding is that users can exert real-time control over the agent's behavior, enhancing the interactivity and adaptability of AI systems in dynamic environments.
Users can now exert real-time control over AI agent behaviors, seamlessly blending style with task performance in complex domains.
Reinforcement learning has proven to be a valuable tool in the creation of advanced AI and robotic systems, contributing to everything from game playing to robotics to foundation models. Through trial-and-error, these AI systems typically learn one, near-optimal behavior to solve their tasks. However, there are many use cases in which one would like to assert some level of control, preferably in real time, over how the task is solved. We refer to these modifications of a core task as styles. We combine universal value function approximators (UVFAs) with carefully selected training scenarios, learning algorithms, and data augmentation to create a framework for coaching agents that exhibit styles in complex domains. We demonstrate the framework's application in the AAA video games Horizon Forbidden West and Gran Turismo, and in an open-source humanoid test domain. Despite the different nature of the domains -- car racing, stylized game combat, and humanoid walking -- each agent shows strong coherence to the style requests while still satisfying the main task in its domain. Importantly, the techniques outlined in this paper allow an end user to choose the final behavior at run time, giving them flexible control over the final executed performance.
