FIVE-VLA: Fast and EffectIVE Autonomous Driving with Recurrent Action Memory

2026-09-16

AI Summary

本文提出FIVE-VLA模型，通过高效的视觉编码器和循环动作记忆模块解决现有自动驾驶模型参数过多、图像处理效率低及缺乏时序记忆的问题。

Abstract

State-of-the-art vision-language-action models (VLA) for autonomous driving face critical limitations: excessive parameter counts, inefficient high-resolution image processing, and lack of temporal memory. We introduce Fast and EffectIVE VLA (FIVE-VLA) to address these through two key contributions. First, we employ an efficient vision encoder that processes high-resolution ($448 \times 896$) images while generating only 98 tokens, over $5\times$ fewer than existing approaches, and bypass text generation entirely for single-pass trajectory prediction. Second, we propose Recurrent Action Memory (RAM), a lightweight module that conditions action prediction on previous action tokens, providing temporal context critical for manoeuvres such as overtaking and emergency braking. With only 641M parameters, FIVE-VLA completes $\sim$10% more routes without traffic rule infractions than the previous state-of-the-art VLA on the challenging Bench2Drive closed-loop driving benchmark. Non-reactive open-loop simulation on the large-scale real-world NVIDIA Physical AI AV dataset shows 10.2% and 7.7% lower collision-violation rates than SimLingo in single- and four-view settings, respectively. Additionally, FIVE-VLA runs at $\sim$30 fps on an A100 and $\sim$4 fps on a T4 GPU (proxy to an edge device), representing an 8-30$\times$ speedup over previous methods.

Problem

• vision-language-action models
• autonomous driving
• parameter counts
• high-resolution image processing
• temporal memory

Innovation

• efficient vision encoder
• Recurrent Action Memory (RAM)
• temporal context