FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving

2026-08-13

AI Summary

This work addresses the challenge that existing vision-language action models based on Group Relative Policy Optimization (GRPO) struggle to obtain effective supervision signals when entire trajectory groups fail. To overcome this, we propose a failure-aware self-evolution framework that identifies groups of trajectories exhibiting low reward and low diversity as failures, triggering a self-distillation mechanism driven by a frozen teacher model. By leveraging hidden observations and answer token-level supervision, our method converts uninterpreted failures into privileged supervisory signals, enabling joint evolution of the policy and failure distribution without reliance on external large models. Evaluated on the nuScenes dataset under G=4 scenarios, our approach reduces the average L2 error from 1.848 meters to 1.500 meters and lowers the persistent failure rate from 13.03% to 11.20%, significantly enhancing the model’s ability to correct rare but severe failures.

Abstract

Reinforcement learning improves autonomous-driving vision-language-action (VLA) models by evaluating trajectories sampled from the current policy. Group relative policy optimization (GRPO) learns from reward differences within each rollout group. When all sampled trajectories are poor, this relative signal can rank failures without identifying behavior outside the failed region. We introduce FIRE-VLA, a failure-informed self-evolution framework that converts such unresolved failures into privileged supervision for the next policy. Low-reward, low-diversity groups trigger self-distillation from a frozen round-start copy of the same model. Teacher and student have the same parameter scale, but only the teacher observes the hidden future trajectory. Supervision follows the student's generated prefix and is restricted to answer tokens, while GRPO remains active for every group. The updated policy supplies the teacher for the next round, allowing the routed failure distribution to change with the policy without requiring a larger external teacher. Starting from the same Qwen2.5-VL-3B SFT checkpoint, the comparison matches student rollout and policy-update counts. On 6,019 examples from 150 held-out nuScenes scenes, FIRE-VLA retains comparable single-sample planning, reduces G=4 mean L2 from 1.848 to 1.500 m, and lowers evaluation-persistent failure prevalence from 13.03% to 11.20%. The reduction in mean error arises mainly from rare severe rollouts rather than uniform improvement across ordinary trajectories.

Problem

• Vision-Language-Action Models
• Autonomous Driving
• Reinforcement Learning
• Failure Recovery
• Policy Optimization

Innovation

• failure-informed learning
• self-evolution
• vision-language-action models
• self-distillation
• autonomous driving