Search papers, labs, and topics across Lattice.
This paper introduces Multi-Objective AI-Informed Preference Learning (MAPL), a novel framework that leverages large language models (LLMs) to derive locomotion rewards from high-level natural language objectives instead of relying on manually crafted reward functions. By prompting the LLM to evaluate trajectories based on semantically meaningful criteria, MAPL effectively captures the multiple competing objectives essential for high-quality locomotion. The approach demonstrates that policies trained with LLM-generated preferences can achieve performance on par with or superior to those utilizing expert-designed rewards across various quadruped locomotion tasks, significantly reducing the need for task-specific reward engineering.
MAPL achieves expert-level locomotion performance without the burden of manual reward design by harnessing the power of language models to evaluate multiple objectives simultaneously.
Reward design remains a major bottleneck in reinforcement learning for robot locomotion, where successful policies often depend on carefully tuned, task-specific reward functions. Preference-based reinforcement learning offers an alternative, but existing LLM-based methods typically ask for a single overall judgment between behaviors, making it difficult to capture the multiple competing objectives that underlie high-quality locomotion. We present Multi-Objective AI-Informed Preference Learning (MAPL), a framework that learns locomotion rewards from high-level natural language objectives rather than manually engineered reward equations. MAPL prompts a large language model to compare trajectories independently along semantically meaningful criteria, using generic language descriptions that are terrain-invariant and require little domain expertise. These objective-wise preferences are used to train a multi-head preference scoring model, whose outputs are aggregated to form a scalar reward for policy optimization. Across four quadruped locomotion environments, MAPL trains policies using only LLM-generated preferences and achieves performance comparable to or better than expert-designed rewards, while eliminating task-specific reward engineering.
