Search papers, labs, and topics across Lattice.
This paper introduces a framework for learning reward functions for robotics that generalize to novel environments by focusing on task-level behavioral invariants rather than pixel-level features. The approach uses a structural reward formulation to encode task strategies and physical constraints, coupled with a hybrid symbolic-numerical procedure to extract these invariants from a small number of demonstrations. Experiments show the learned rewards achieve better process alignment and policy rollout ranking compared to baselines, and generalize zero-shot to variations in position, viewpoint, and objects in real-world manipulation tasks.
Forget painstakingly engineering reward functions for every new robotic setup – this method learns a single, reusable reward representation from just five demonstrations that generalizes zero-shot to variations in object, viewpoint, and position.
Designing reward functions that generalize beyond controlled laboratory settings remains a fundamental challenge in reinforcement learning for robotics. In open-world manipulation problems, a single task can appear in numerous variants through different object instances, positions, and camera viewpoints. Recent vision-based reward models tend to memorize specific pixel distributions and fail to generalize beyond their training conditions. To address this, we propose a framework that learns invariant symbolic reward functions from as few as five demonstrations. The insight is to shift from visual feature-fitting to the discovery of behavioral invariants: task-level properties that remain constant across diverse visual instantiations. The framework has two coupled components: a structural reward formulation that encodes task-level strategies and physical constraints while preserving optimal policy invariance, and a hybrid symbolic-numerical procedure that distills these invariants from demonstrations without online interaction. Experiments on eight Meta-World tasks and three Franka manipulation tasks demonstrate that our method achieves stronger process alignment and policy rollout ranking abilities compared to baselines, accelerating downstream policy learning. Three real-world out-of-distribution experiments further show that the same learned reward generalizes zero-shot to position, viewpoint, and object variations, enabling a single reward representation to be reused across diverse task variants in practice.
