FastDSAC: Enhancing Policy Plasticity via Constrained Exploration for Scalable Humanoid Locomotion
This work addresses the instability in value estimation and degradation of policy plasticity commonly observed in high-throughput sampling scenarios with frequent data updates. To mitigate these issues, the authors propose FastDSAC, an algorithm built upon a distributed Actor-Critic framework that models the policy using a truncated Gaussian distribution to simultaneously respect action constraints and preserve exploratory stochasticity. The method incorporates an adaptive variance modulation mechanism to enhance the accuracy of value estimation and employs implicit regularization to maintain the adaptability of the policy network. Experimental results demonstrate that FastDSAC achieves more stable training dynamics, faster convergence, and superior asymptotic performance compared to existing approaches on both the MuJoCo Playground and HumanoidBench benchmarks.