Dynamic Control Barrier Function Regulation with Vision-Language Models for Safe, Adaptive, and Realtime Visual Navigation

2026-03-22

AI Summary

This work addresses the challenge of safe navigation in perception-constrained, dynamic, and unstructured environments, where conventional control barrier functions (CBFs) with fixed parameters struggle to adapt to varying environmental risks. The authors propose AlphaAdj, a novel framework that integrates vision-language models (VLMs) with CBFs for the first time, leveraging first-person RGB images to estimate semantic risk in real time and dynamically adjust CBF parameters to balance safety and efficiency. To mitigate VLM inference latency, a lightweight asynchronous fusion mechanism is introduced, incorporating geometric velocity-aware upper bounds and staleness gating. Experimental results demonstrate that AlphaAdj achieves efficient, collision-free navigation across diverse scenarios with both static and dynamic obstacles, improving path efficiency by up to 18.5% over fixed-parameter CBFs while significantly enhancing task success rate and robustness.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Robots operating in dynamic, unstructured environments must balance safety and efficiency under potentially limited sensing. While control barrier functions (CBFs) provide principled collision avoidance via safety filtering, their behavior is often governed by fixed parameters that can be overly conservative in benign scenes or overly permissive near hazards. We present AlphaAdj, a vision-to-control navigation framework that uses egocentric RGB input to adapt the conservativeness of a CBF safety filter in real time. A vision-language model(VLM) produces a bounded scalar risk estimate from the current camera view, which we map to dynamically update a CBF parameter that modulates how strongly safety constraints are enforced. To address asynchronous inference and non-trivial VLM latency in practice, we combine a geometric, speed-aware dynamic cap and a staleness-gated fusion policy with lightweight implementation choices that reduce end-to-end inference overhead. We evaluate AlphaAdj across multiple static and dynamic obstacle scenarios in a variety of environments, comparing against fixed-parameter and uncapped ablations. Results show that AlphaAdj maintains collision-free navigation while improving efficiency (in terms of path length and time to goal) by up to 18.5% relative to fixed settings and improving robustness and success rate relative to an uncapped baseline.

Problem

• control barrier functions
• visual navigation
• dynamic environments
• safety-efficiency trade-off
• real-time adaptation

Innovation

• Dynamic Control Barrier Function
• Vision-Language Model
• Real-time Adaptive Navigation
• Risk-aware Safety Filtering
• Egocentric Visual Navigation