Infant Spontaneous Movement Noise Improves Exploration in Deep RL

2026-06-15

AI Summary

This work addresses the limitations of conventional deep reinforcement learning exploration strategies, which rely on temporally uncorrelated white noise and struggle to efficiently cover the state space. Inspired by the temporally structured noise observed in infants’ spontaneous movements, this study introduces—for the first time—the statistical characteristics of human motor development into reinforcement learning. It proposes a dynamically scheduled colored exploration noise mechanism that progressively enhances the temporal autocorrelation of exploration noise throughout training, guided by power spectral density modeling. Empirical evaluations across multiple reinforcement learning environments demonstrate that this approach yields more structured exploration behaviors and substantially improved sample efficiency, significantly outperforming existing exploration strategies.

Abstract

Exploration in deep reinforcement learning (RL) is commonly implemented as temporally uncorrelated white noise. However, recent works show that temporally correlated colored noise can improve exploration efficiency by producing smooth trajectories with better coverage of the state space. We inquire whether action noise inspired by infant spontaneous movements can also improve exploration in deep RL. We find that the power spectral densities of babies' end-effector velocities follow a colored noise process where the spectral exponent increases with age. Inspired by this developmental pattern, we introduce a mechanism that progressively increases the temporal auto-correlation of exploration noise during RL training, matching the infant statistics. Experiments across several RL environments show that infant-inspired noise produces structured exploratory behavior and can improve learning efficiency compared to conventional exploration strategies. These findings suggest that human motor and cognitive development can provide useful guidance for designing learning mechanisms in artificial agents. Our code is available at https://github.com/trieschlab/baby-noise-rl.

Problem

• exploration
• deep reinforcement learning
• colored noise
• infant spontaneous movement
• temporal correlation

Innovation

• infant spontaneous movement
• colored noise
• exploration in deep RL
• temporal auto-correlation
• developmental inspiration