Search papers, labs, and topics across Lattice.
This paper introduces Dream2Reward, a novel approach for generating dense rewards in robotic manipulation by learning a language-conditioned successful latent transition field from positive demonstrations. The method evaluates the quality of transitions by comparing the observed displacement against predicted successful outcomes, effectively penalizing incorrect movements even when they superficially appear to show progress. The results demonstrate that Dream2Reward outperforms traditional progress-based reward models, providing clearer success-failure distinctions and enhancing performance in both online and offline policy learning scenarios, including real-robot applications.
Transition-level comparisons in Dream2Reward reveal that even subtle missteps in robotic motion can be effectively penalized, leading to significantly improved learning outcomes.
Learning robotic policies requires dense rewards that remain informative when behavior departs from successful demonstrations. Progress-based rewards estimate how far an observation has advanced along a nominal successful trajectory, but may remain high after an incorrect transition. We introduce Dream2Reward, which learns a language-conditioned successful latent transition field from positive demonstrations. Given the visual history up to a transition start, the model predicts the latent displacement associated with successful execution and scores the observed displacement through signed directional and symmetric magnitude agreement. This transition-level comparison penalizes wrong-direction, overshooting, and stagnant motion even when the resulting observation appears to show progress. Dream2Reward requires no failure annotations, progress labels, or synthetic negatives, and produces a dense causal reward. Across mechanism diagnostics and shared-trajectory evaluations, it provides stronger success-failure separation and more informative feedback on low-quality behavior than progress-based alternatives. Across online and offline policy learning, the same frozen reward model reduces reward hacking and supports stronger downstream performance, including in real-robot manipulation. These results show that comparing realized motion with predicted successful change provides an effective way to convert positive demonstrations into dense rewards for robot learning.
