Search papers, labs, and topics across Lattice.
This paper explores the use of Vision Language Models (VLMs) to annotate a video game dataset, enabling the extraction of human-defined rewards for training reinforcement learning (RL) agents. By leveraging offline RL, the authors demonstrate that agents can be conditioned to respond to these annotated rewards, simplifying the traditionally complex process of reward identification and weighting. The findings reveal both the potential and challenges of this approach, highlighting limitations encountered during initial experiments.
Annotating video game datasets with VLMs can transform the way RL agents learn by simplifying reward extraction and conditioning.
Reinforcement Learning (RL) is a powerful but far from easy-to-use technique for policy learning. In the specific case of video games, access to the game engine is required to get rewards for training (e.g. to collect rewards from the environment). Furthermore, the proper identification and weighting of the rewards generally requires a difficult trial-and-error approach. Lastly, rewards are often sparse and understanding how they eventually affect the learned policy is a non-trivial exercise. To ease these issues we propose annotating a video game dataset with Vision Language Models (VLMs) instructed to extract human defined rewards. We show that offline RL can then be used to train a conditioned agent that responds accordingly to the desired returns and we discuss the difficulties and limitations that emerged in our early experiments.
NVIDIA