Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications

2026-07-01

AI Summary

This work addresses the limitations of conventional reinforcement learning approaches for quadrupedal locomotion, which rely on handcrafted Markovian reward functions that lack explicit control over gait patterns and offer poor policy interpretability. The study introduces, for the first time, a parameterized Signal Temporal Logic (STL) template to formally specify gait constraints across varying speeds. A smooth approximation of STL robustness is leveraged to construct a dense, continuous reward function that unifies multiple objectives—including safety bounds, gait synchronization, and command tracking—into a single optimization framework. Evaluated on the Barkour quadruped platform using PPO, MuJoCo XLA simulation, domain randomization, and parallelized training, the proposed method demonstrates significantly improved velocity-tracking accuracy and training stability compared to manually designed rewards.

Abstract

Reinforcement learning (RL) for quadruped locomotion commonly depends on fixed, hand-crafted, and Markovian reward functions that limit both interpretability of learned policies and lack explicit control over gait behaviors. We introduce a framework where distinct gaits are specified using parameterized constraints expressed in Signal Temporal Logic (STL). These include safety bounds, gait synchronization constraints, command tracking, and actuation bounds. From these specifications, we develop a reward shaping mechanism that provides learning agents a dense, continuous reward landscape that encodes desired behavior. We define parametric STL templates for three speed regimes (walking-trot, trot, bound), calibrate their parameters from reference rollouts, and compute rewards from using smooth approximations of STL robustness over the rollouts. The generated rewards can be used to provide shaped gradients compatible with Proximal Policy Optimization (PPO). We instantiate the approach on Google's Barkour quadruped robot in MuJoCo XLA (MJX). We use parallelization within the simulator to improve training speeds and use domain randomization to robustify learned policies. We show that compared to a baseline of hand-crafted rewards, the STL-shaped rewards yield tighter velocity tracking and more stable training. Videos can be found on our project website: https://stl-locomotion.github.io/.

Problem

• quadruped locomotion
• reinforcement learning
• gait control
• reward function
• temporal logic

Innovation

• Signal Temporal Logic
• quadruped locomotion
• reward shaping
• gait specification
• reinforcement learning