Feature Extractor or Decision Maker: Rethinking the Role of Visual Encoders in Visuomotor Policies

2024-09-30

arXiv.org

AI Summary

This work challenges the conventional assumption that visual encoders serve solely as “feature extractors” in end-to-end visuomotor policies, investigating whether they actively participate in decision-making. We propose the **Visual Alignment Testing** framework—a novel empirical methodology—to assess the intrinsic decision-making capacity of visual encoders. Our experiments demonstrate, for the first time, that end-to-end trained visual encoders possess significant task-relevant decision-making capability; however, this capacity degrades markedly when encoders are initialized with out-of-distribution (OOD) pretraining, leading to an average 42% drop in motor control performance. These findings reveal that under motor supervision, visual encoders actively learn task semantics and jointly fulfill both feature representation and decision-making roles. The results establish a new paradigm for designing task-conditioned, context-aware visual encoders and provide foundational theoretical support for rethinking encoder architecture and training objectives in embodied AI.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

An end-to-end (E2E) visuomotor policy is typically treated as a unified whole, but recent approaches using out-of-domain (OOD) data to pretrain the visual encoder have cleanly separated the visual encoder from the network, with the remainder referred to as the policy. We propose Visual Alignment Testing, an experimental framework designed to evaluate the validity of this functional separation. Our results indicate that in E2E-trained models, visual encoders actively contribute to decision-making resulting from motor data supervision, contradicting the assumed functional separation. In contrast, OOD-pretrained models, where encoders lack this capability, experience an average performance drop of 42% in our benchmark results, compared to the state-of-the-art performance achieved by E2E policies. We believe this initial exploration of visual encoders' role can provide a first step towards guiding future pretraining methods to address their decision-making ability, such as developing task-conditioned or context-aware encoders.

Problem

• Evaluating functional separation in visuomotor policy components
• Assessing visual encoders' role in decision-making under motor supervision
• Addressing performance gap between E2E and OOD-pretrained models

Innovation

• Proposes Visual Alignment Testing framework
• Shows visual encoders aid decision-making
• Suggests task-conditioned encoder development