Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces GORDON, a novel graph-based framework for learning object-centric rewards from action-free video demonstrations, addressing the challenges of sparse rewards and manual subtask annotation in long-horizon manipulation tasks. By representing visual scenes as graphs of detected objects and their spatial relationships, GORDON employs a graph neural network to create a task-aligned latent space that facilitates the computation of dense rewards based on the proximity to demonstrated goal configurations. The framework not only enhances performance in short-horizon settings but also enables automatic subtask discovery and policy learning in complex long-horizon tasks, achieving a notable average success rate of 74.4% across various benchmarks.
GORDON achieves a remarkable 74.4% success rate in long-horizon manipulation tasks by automatically discovering subtasks and learning dense object-centric rewards from video demonstrations.
Learning long-horizon manipulation skills with reinforcement learning remains challenging due to the complexity of reward design, the limited guidance of sparse rewards, and the high cost of manual subtask annotation. Visual demonstrations can provide supervision for reward learning, but rewards learned from raw pixels can be brittle and sensitive to visual variation, background appearance, and robot motion. In this work, we propose GORDON, a graph-based object-centric reward learning framework that learns dense rewards from action-free video demonstrations. Each visual scene is represented as a graph of detected objects and spatial relations, and a graph neural network is trained in a self-supervised manner to embed these graphs into a task-aligned latent space. To align the representation with semantic task progress, we introduce an activity-aware weighted pooling mechanism that emphasizes task-relevant objects while masking robot-dominated motion. The dense reward is then computed as distances in the learned latent space of the current state to demonstrated goal configurations, providing a measure of task progress. In long-horizon tasks, the temporal profile of this reward reveals stage-wise object-state transitions, enabling automatic subtask discovery without manual segmentation. The discovered segments are then used to train subtask-specific rewards and specialized policies that are composed sequentially. Experiments on seven manipulation tasks on MAGICAL and ManiSkill3 benchmarks show that our object-centric reward improves reinforcement learning in short-horizon settings and enables successful policy learning in complex long-horizon tasks through automatic decomposition, achieving an average success rate of 74.4% across the long-horizon tasks (on average approximately +35 p.p. vs. best learned baseline and approximately +25 p.p. vs. oracle).