TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotion

2025-03-26

AI Summary

Quadrupedal robot reinforcement learning suffers from teacher-student representation misalignment, covariate shift induced by behavioral cloning, and poor deployment adaptability—leading to weak real-world generalization. Method: We propose a contrastive learning-based teacher-aligned representation framework that, for the first time, incorporates privileged teacher information into self-supervised contrastive learning to achieve representation alignment between teacher and student in simulation. The framework supports online adaptation and continual fine-tuning without privileged signals during deployment. Contribution/Results: By integrating contrastive learning, representation alignment, and behavioral cloning, our method accelerates training by 2× while improving average out-of-distribution (OOD) generalization performance by 40%—surpassing even the full-privilege teacher policy. It significantly enhances real-world deployment capability and overcomes the generalization bottleneck inherent in conventional behavioral cloning approaches.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Behavior Learning & Control
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Quadrupedal locomotion via Reinforcement Learning (RL) is commonly addressed using the teacher-student paradigm, where a privileged teacher guides a proprioceptive student policy. However, key challenges such as representation misalignment between the privileged teacher and the proprioceptive-only student, covariate shift due to behavioral cloning, and lack of deployable adaptation lead to poor generalization in real-world scenarios. We propose Teacher-Aligned Representations via Contrastive Learning (TAR), a framework that leverages privileged information with self-supervised contrastive learning to bridge this gap. By aligning representations to a privileged teacher in simulation via contrastive objectives, our student policy learns structured latent spaces and exhibits robust generalization to Out-of-Distribution (OOD) scenarios, surpassing the fully privileged"Teacher". Results showed accelerated training by 2x compared to state-of-the-art baselines to achieve peak performance. OOD scenarios showed better generalization by 40 percent on average compared to existing methods. Additionally, TAR transitions seamlessly into learning during deployment without requiring privileged states, setting a new benchmark in sample-efficient, adaptive locomotion and enabling continual fine-tuning in real-world scenarios. Open-source code and videos are available at https://ammousa.github.io/TARLoco/.

Problem

• Aligns teacher-student representations for better generalization
• Reduces covariate shift in quadrupedal locomotion learning
• Enables deployable adaptation without privileged information

Innovation

• Aligns student-teacher representations via contrastive learning
• Enhances generalization with structured latent spaces
• Enables deployable adaptation without privileged states