Quadruped Parkour Learning: Sparsely Gated Mixture of Experts with Visual Input

2026-04-21

AI Summary

This work addresses the challenge of balancing obstacle negotiation capability and computational efficiency for vision-driven quadrupedal robots traversing complex terrains such as high steps. We propose, for the first time, integrating a Sparsely Gated Mixture-of-Experts (MoE) architecture into parkour control policies. By activating only a subset of parameters during inference while maintaining a constant number of active parameters, our approach significantly outperforms conventional MLP-based policies. Real-world experiments on the Unitree Go2 platform demonstrate that the proposed method doubles the success rate of obstacle traversal. In contrast, a standard MLP requires a 14.3% increase in computation time to achieve comparable performance, revealing a novel trade-off between model scaling and computational overhead.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Search and Optimization: Learning to Search
• Computer Vision: Multi-modal Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Robotic parkour provides a compelling benchmark for advancing locomotion over highly challenging terrain, including large discontinuities such as elevated steps. Recent approaches have demonstrated impressive capabilities, including dynamic climbing and jumping, but typically rely on sequential multilayer perceptron (MLP) architectures with densely activated layers. In contrast, sparsely gated mixture-of-experts (MoE) architectures have emerged in the large language model domain as an effective paradigm for improving scalability and performance by activating only a subset of parameters at inference time. In this work, we investigate the application of sparsely gated MoE architectures to vision-based robotic parkour. We compare control policies based on standard MLPs and MoE architectures under a controlled setting where the number of active parameters at inference time is matched. Experimental results on a real Unitree Go2 quadruped robot demonstrate clear performance gains, with the MoE policy achieving double the number of successful trials in traversing large obstacles compared to a standard MLP baseline. We further show that achieving comparable performance with a standard MLP requires scaling its parameter count to match that of the total MoE model, resulting in a 14.3\% increase in computation time. These results highlight that sparsely gated MoE architectures provide a favorable trade-off between performance and computational efficiency, enabling improved scaling of control policies for vision-based robotic parkour. An anonymized link to the codebase is https://osf.io/v2kqj/files/github?view_only=7977dee10c0a44769184498eaba72e44.

Problem

• quadruped parkour
• vision-based locomotion
• sparse gating
• mixture of experts
• computational efficiency

Innovation

• Mixture of Experts
• Sparse Gating
• Vision-based Locomotion
• Quadrupedal Parkour
• Computational Efficiency