Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models

2026-07-12

AI Summary

This work addresses the susceptibility of robotic foundation models to shortcut learning in real-world scenarios, where policies often rely on spurious scene-level correlations rather than task-relevant visual evidence. To mitigate this, the authors propose an Artificial Foveal Perception (AFP) module that, during fine-tuning, generates task-conditioned saliency masks as auxiliary supervision signals to guide the policy toward task-critical visual regions—without altering the original architecture. AFP introduces, for the first time, a task-conditioned foveal mechanism into robotic foundation models through a lightweight, policy-agnostic vision-language processing pipeline and an alignment loss function. Experiments demonstrate that AFP significantly enhances model robustness and data efficiency, reduces fine-tuning time, mitigates overfitting across multiple state-of-the-art models, and improves generalization under environmental perturbations.

Abstract

Robotic foundation models have recently made substantial progress in multi-task capability, cross-embodiment transfer, and language-conditioned control. Yet robust deployment across diverse real-world settings remains difficult, in part because policies often fail to distinguish causally relevant visual structure from spurious scene-level correlations. We identify this failure mode as shortcut learning: the tendency to exploit predictive but non-causal correlations in the training distribution rather than the task-relevant visual evidence that determines successful action. Although shortcut learning has been extensively studied in computer vision and broader machine learning, its role in robotic foundation models remains comparatively underexplored. We propose Artificial Foveated Perception (AFP), a lightweight, policy-agnostic module that takes the same vision and language inputs as Vision-Language-Action and World Action Model pipelines and predicts task-conditioned masks over relevant objects, the robot, and other action-critical regions. We use these masks primarily as an auxiliary grounding signal during fine-tuning, aligning policy attention with task-relevant regions while leaving the core architecture unchanged. After fine-tuning, the policy executes on the original observation stream without requiring AFP in the control loop. We evaluate AFP across state-of-the-art robotic foundation models and show that foveated perception reduces fine-tuning time, suppresses overfitting, and improves generalization under environmental perturbations. Ablations over mask quality and grounding-loss design further show that these gains arise from directing policy learning toward task-relevant visual evidence. These results suggest that task-conditioned foveated perception is a practical mechanism for making robotic foundation models more robust, data-efficient, and scalable.

Problem

• shortcut learning
• robotic foundation models
• visual grounding
• causal perception
• generalization

Innovation

• Artificial Foveated Perception
• shortcut learning
• robotic foundation models
• task-conditioned attention
• visual grounding