Rethinking Visual Embodiment Dependence in Visuomotor Policies

2026-09-15

AI Summary

研究视觉体现依赖性问题，通过3D点云中的体现规范化和配置去相关增强方法，改善人到机器人的策略转移，提高视觉运动学习的鲁棒性。

Abstract

Visuomotor policies observe both the task scene and the acting embodiment, allowing embodiment-specific visual cues to influence action prediction. We study this phenomenon as visual embodiment dependence (VED) and show, through cue-conflict interventions across representative policies, that visible robot configuration can become a shortcut to task progress. Rather than eliminating VED, we argue that it should be structured around embodiment information that supports control and generalization. We realize this through embodiment canonicalization in 3D point clouds, replacing the original embodiment with a canonical end-effector representation (CER) that preserves control-relevant geometry while abstracting embodiment-specific morphology. Its editable form further enables configuration-decorrelation augmentation for unfamiliar robot configurations. Experiments show that embodiment canonicalization substantially improves human-to-robot policy transfer without robot demonstrations, while simply removing the embodiment is insufficient without preserving control-relevant geometry. We further find that CER itself can become a configuration shortcut when robot configuration becomes decoupled from task progress; configuration-decorrelation augmentation mitigates this failure mode and restores robust recovery without sacrificing performance on seen configurations. Together, these results show that robust visuomotor learning benefits from structuring, rather than removing, visual embodiment information. Project website: https://tonyfang.net/ved

Problem

• Visual Embodiment Dependence
• Visuomotor Policies
• Embodiment Canonicalization

Innovation

• Visual Embodiment Dependence (VED)
• Embodiment Canonicalization
• Canonical End-Effector Representation (CER)
• Configuration-Decorrelation Augmentation