Event-Adaptive Motion Planning with Distilled Vision-Language Model in Safety-Critical Situations

2026-06-24

AI Summary

This work addresses the challenge of ensuring collision-free navigation for robots in safety-critical environments, where unpredictable behaviors of dynamic agents complicate the simultaneous achievement of semantic understanding and real-time control. To this end, the authors propose an Event-Adaptive Motion Planning (EAMP) framework that selectively activates a lightweight vision-language model (SemNav-VLM) through a configurable prompt-based semantic event triggering mechanism. By integrating physics-informed verification distillation with semantic model predictive control (SMPC), EAMP enables, for the first time, efficient co-design between VLM inference and real-time motion planning. The approach significantly enhances dynamic safety margins, outperforming existing baselines in logistics scenarios while maintaining real-time computational performance.

Abstract

Robot navigation in safety-critical scenarios faces significant challenges from unforeseen semantic events, where collisions arise primarily from the unpredictable behaviors of dynamic agents rather than unseen objects. While large vision-language models (VLMs) offer remarkable capabilities in commonsense reasoning, frequently invoking them within the continuous control loop introduces severe computational latency, fundamentally destabilizing physical execution. To address these challenges, we propose event-adaptive motion planning (EAMP), an efficient framework for VLM-based robot navigation. Specifically, a prompt-configurable semantic event trigger (PC-SET) selectively activates semantic intervention by continuously monitoring short temporal clips for behavioral anomalies. Upon triggering, an event-triggered distilled SemNav-VLM, fine-tuned via physically verified semantic distillation, maps detected anomalies into discrete strategy-level decisions. Subsequently, a semantic model predictive control (SMPC) module translates these strategies into dynamic reconfigurations of optimization objectives and geometric references. Extensive experiments in safety-critical logistics scenarios demonstrate that EAMP effectively aligns high-level reasoning with low-level control, significantly improving dynamic safety margins over existing baselines while preserving real-time efficiency.

Problem

• safety-critical navigation
• semantic events
• dynamic agents
• computational latency
• vision-language models

Innovation

• event-adaptive motion planning
• vision-language model distillation
• semantic event trigger
• semantic model predictive control
• safety-critical navigation