Hierarchical Vision Language Action Model Using Success and Failure Demonstrations

2025-12-03

AI Summary

Existing Vision-Language-Action (VLA) models exclusively leverage successful demonstration data, overlooking critical information about policy fragility embedded in failed execution attempts. Method: We propose VINE, the first VLA framework to incorporate failure-aware feasibility tree search during planning. It employs a 2D scene graph to jointly predict subgoals and prune fragile execution paths, transforming mixed-quality (success/failure) teleoperation data into structured learning signals. VINE adopts a hierarchical architecture: a high-level semantic reasoning module operates over the scene graph, while a low-level layer reuses pretrained skill modules—requiring no modification to the action execution layer. Contribution/Results: Under offline training, VINE significantly improves success rate and robustness on complex manipulation tasks, empirically validating that failure data is essential for enhancing the real-world deployability of VLA models.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Visual Reasoning & Symbolic Representations
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Prior Vision-Language-Action (VLA) models are typically trained on teleoperated successful demonstrations, while discarding numerous failed attempts that occur naturally during data collection. However, these failures encode where and how policies can be fragile, information that can be exploited to improve robustness. We address this problem by leveraging mixed-quality datasets to learn failure-aware reasoning at planning time. We introduce VINE, a hierarchical vision-language-action model that separates high-level reasoning (System 2) from low-level control (System 1) under a hierarchical reinforcement learning formalism, making failures usable as a structured learning signal rather than noisy supervision. System 2 performs feasibility-guided tree search over a 2D scene-graph abstraction: it proposes subgoal transitions, predicts success probabilities from both successes and failures, and prunes brittle branches before execution, effectively casting plan evaluation as feasibility scoring. The selected subgoal sequence is then passed to System 1, which executes low-level actions without modifying the agent's core skills. Trained entirely from offline teleoperation data, VINE integrates negative experience directly into the decision loop. Across challenging manipulation tasks, this approach consistently improves success rates and robustness, demonstrating that failure data is an essential resource for converting the broad competence of VLAs into robust execution.

Problem

• Leverages mixed-quality datasets for failure-aware reasoning
• Separates high-level reasoning from low-level control hierarchically
• Uses failure data to improve success rates and robustness

Innovation

• Hierarchical model separates reasoning from control
• Feasibility-guided tree search with success/failure probabilities
• Integrates negative experience into offline decision loop