Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Forward CIP (F-CIP), an RL-native intrinsic motivation objective derived solely from system dynamics, eliminating the need for human-selected information variables. By proving F-CIP's compatibility with RL and demonstrating its efficacy with existing algorithms, the authors show it enables unsupervised discovery of fundamental behaviors like balancing and controllability. When combined with a simple forward-velocity reward, F-CIP facilitates the learning of complex gaits such as hopping and running without explicit reward engineering.
Intrinsic motivation can be learned from system dynamics alone, unlocking unsupervised discovery of essential robotic behaviors and complex locomotion.
Reinforcement learning (RL) is a powerful paradigm for training agents, yet its success rests on domain expertise of human engineers who design informative reward signals for every new task. Unsupervised RL aims to reduce this engineering with intrinsic motivation (IM): reward signals that emerge from the agent environment interaction itself. Existing IM objectives, however, involve the selection of information variables, which re-introduces domain expertise the field has sought to eliminate. We introduce Forward CIP (F-CIP), an RL-native formulation of the Controllable Information Production (CIP) objective, which is defined by the system's dynamics alone and requires no such selection. We prove that F-CIP is compatible with RL and demonstrate its effectiveness with existing algorithms. Training agents with F-CIP results in unsupervised discovery of primitive behaviors such as balancing and maintaining controllability, which are essential for more complex robot behaviors. Paired with a simple forward-velocity reward, our method produces coordinated gaits such as hopping and running which otherwise require reward engineering to learn.