Search papers, labs, and topics across Lattice.
This paper introduces VLM-AR3L, a novel framework that utilizes Vision-Language Models to generate both absolute and relative rewards for reinforcement learning in complex environments. By interpreting visual observations in the context of natural language task goals, VLM-AR3L effectively learns reward signals that enhance agent performance in tasks with abstract objectives. Experimental evaluations demonstrate that VLM-AR3L significantly outperforms existing VLM-based reward learning methods across various benchmarks, particularly in challenging scenarios like Minecraft.
VLM-AR3L outperforms traditional reward learning methods by integrating absolute and relative rewards, enabling agents to navigate complex tasks with greater efficacy.
Designing effective reward functions remains a major challenge in reinforcement learning (RL), particularly in open-ended environments where task goals are abstract and difficult to quantify. In this work, we present VLM-AR3L, a framework that leverages Vision-Language Models (VLMs) to provide both absolute and relative rewards for RL. VLM-AR3L interprets an agent's visual observations in the context of a natural language task goal, and learns both absolute and relative rewards from VLM-generated preference labels. The absolute reward model predicts scalar evaluations for individual states, while the relative reward model compares consecutive observations to infer progress or regression toward the task goal. Their integration combines the stability of state-based evaluation with the robustness of comparative supervision. We evaluate VLM-AR3L across benchmarks spanning classic control, manipulation, and open-world embodied tasks, with a particular focus on Minecraft given its visual complexity and long-horizon decision-making requirements. Experimental results show that VLM-AR3L consistently outperforms prior VLM-based reward learning methods.
NVIDIA