Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models

2025-12-28

AI Summary

Manually designing reward functions for high-dimensional musculoskeletal motor control—especially from natural-language goals (e.g., “walk forward upright”)—is notoriously difficult due to the semantic gap between abstract instructions and low-level biomechanical dynamics. Method: We propose MoVLR, the first framework to integrate vision-language models (VLMs) into embodied reward learning. MoVLR aligns linguistic goal descriptions with physical motion representations via VLMs, then jointly optimizes task-specific rewards through closed-loop iteration of reinforcement learning and biomechanical dynamics modeling. Contribution/Results: MoVLR eliminates reliance on expert domain knowledge, enabling end-to-end mapping from semantic goals to implicit physiological coordination policies. Evaluated across diverse locomotion and manipulation tasks, the learned rewards significantly improve motion coordination, goal fidelity, and cross-task generalization—demonstrating VLMs’ efficacy as principled anchors for physiological movement priors.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Language and Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Discovering effective reward functions remains a fundamental challenge in motor control of high-dimensional musculoskeletal systems. While humans can describe movement goals explicitly such as "walking forward with an upright posture," the underlying control strategies that realize these goals are largely implicit, making it difficult to directly design rewards from high-level goals and natural language descriptions. We introduce Motion from Vision-Language Representation (MoVLR), a framework that leverages vision-language models (VLMs) to bridge the gap between goal specification and movement control. Rather than relying on handcrafted rewards, MoVLR iteratively explores the reward space through iterative interaction between control optimization and VLM feedback, aligning control policies with physically coordinated behaviors. Our approach transforms language and vision-based assessments into structured guidance for embodied learning, enabling the discovery and refinement of reward functions for high-dimensional musculoskeletal locomotion and manipulation. These results suggest that VLMs can effectively ground abstract motion descriptions in the implicit principles governing physiological motor control.

Problem

• Discovering effective reward functions for high-dimensional musculoskeletal systems.
• Bridging the gap between goal specification and movement control.
• Transforming language and vision assessments into structured guidance for embodied learning.

Innovation

• Leveraging vision-language models for reward function discovery
• Iterative exploration of reward space via VLM feedback
• Transforming language vision assessments into structured guidance