Search papers, labs, and topics across Lattice.
This paper investigates the impact of infant-inspired spontaneous movement noise on exploration in deep reinforcement learning (RL) by introducing a mechanism that increases the temporal auto-correlation of exploration noise during training. The authors find that this approach, which mimics the developmental patterns of infants' end-effector velocities, leads to more structured exploratory behavior and enhances learning efficiency across various RL environments. The results indicate that leveraging insights from human motor development can significantly inform the design of exploration strategies in artificial agents.
Infant-inspired noise can enhance exploration efficiency in deep RL, outperforming traditional uncorrelated noise strategies.
Exploration in deep reinforcement learning (RL) is commonly implemented as temporally uncorrelated white noise. However, recent works show that temporally correlated colored noise can improve exploration efficiency by producing smooth trajectories with better coverage of the state space. We inquire whether action noise inspired by infant spontaneous movements can also improve exploration in deep RL. We find that the power spectral densities of babies' end-effector velocities follow a colored noise process where the spectral exponent increases with age. Inspired by this developmental pattern, we introduce a mechanism that progressively increases the temporal auto-correlation of exploration noise during RL training, matching the infant statistics. Experiments across several RL environments show that infant-inspired noise produces structured exploratory behavior and can improve learning efficiency compared to conventional exploration strategies. These findings suggest that human motor and cognitive development can provide useful guidance for designing learning mechanisms in artificial agents. Our code is available at https://github.com/trieschlab/baby-noise-rl.
