LANTERN: A Closed-Loop Benchmark for VLM-Based Cooperative Driving with Temporally Grounded Warnings

2026-09-06

AI Summary

LANTERN通过分离和匹配警告与无警告情况下的物理事件，评估基于视觉语言模型的驾驶中时间定位警告的有效性，提高驾驶安全性。

Abstract

We present LANTERN, a closed-loop benchmark for temporally grounded cooperative warnings. LANTERN separates warning onset, hazard onset, warning termination, and post-hazard recovery, and evaluates each physical event under matched warning and no-warning executions so that the warning's contribution is measured in isolation rather than confounded with onboard vision. The benchmark spans six safety-critical scenario families and provides 3,272 sequences with 236,309 frames for training, together with 120 matched route pairs for closed-loop evaluation. Each hazard route is evaluated under the warning and no-warning conditions, while its no-hazard control penalizes unconditional braking. We further introduce the Cooperative Unified Score (CUS), a safety-gated metric that jointly rewards route progress, anticipation, clearance, and recovery. Fine-tuning a representative VLM driving model raises CUS from 34.6 without warnings to 75.5 with them, demonstrating both the value of cooperative warnings and the discriminative power of the paired protocol. All resources will be made publicly available.

Problem

• temporally grounded warnings
• cooperative driving
• closed-loop benchmark
• safety-critical scenarios
• visual language model

Innovation

• closed-loop benchmark
• temporally grounded warnings
• Cooperative Unified Score (CUS)
• VLM-based cooperative driving