VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer

2025-12-09

AI Summary

Deploying Vision-Language-Action (VLA) models safely in unstructured environments—while executing multi-task instructions and avoiding physical collisions—remains a critical challenge. Method: We propose AEGIS, a plug-and-play safety architecture that integrates Control Barrier Functions (CBFs) into the VLA pipeline for the first time, enabling a provably safe, modular constraint layer that decouples task performance from formal safety guarantees. Concurrently, we introduce SafeLIBERO—the first safety-critical benchmark for VLA evaluation. Results: Experiments demonstrate that AEGIS improves obstacle avoidance rate by 59.16% and task success rate by 17.25%, without modifying the underlying VLA model. All code, pretrained models, and the SafeLIBERO dataset are publicly released.

Technology Category

• Computer Vision: Language and Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in generalizing across diverse robotic manipulation tasks. However, deploying these models in unstructured environments remains challenging due to the critical need for simultaneous task compliance and safety assurance, particularly in preventing potential collisions during physical interactions. In this work, we introduce a Vision-Language-Safe Action (VLSA) architecture, named AEGIS, which contains a plug-and-play safety constraint (SC) layer formulated via control barrier functions. AEGIS integrates directly with existing VLA models to improve safety with theoretical guarantees, while maintaining their original instruction-following performance. To evaluate the efficacy of our architecture, we construct a comprehensive safety-critical benchmark SafeLIBERO, spanning distinct manipulation scenarios characterized by varying degrees of spatial complexity and obstacle intervention. Extensive experiments demonstrate the superiority of our method over state-of-the-art baselines. Notably, AEGIS achieves a 59.16% improvement in obstacle avoidance rate while substantially increasing the task execution success rate by 17.25%. To facilitate reproducibility and future research, we make our code, models, and the benchmark datasets publicly available at https://vlsa-aegis.github.io/.

Problem

• Enhancing robot safety in unstructured environments with plug-and-play safety layers
• Preventing collisions during manipulation tasks while maintaining task compliance
• Integrating safety constraints into Vision-Language-Action models with theoretical guarantees

Innovation

• Plug-and-play safety layer using control barrier functions
• Integrates with existing models to ensure safety with theoretical guarantees
• Achieves significant improvement in obstacle avoidance and task success