Contextual Safety Reasoning and Grounding for Open-World Robots

2026-02-23

AI Summary

This work addresses the limitations of traditional robot safety approaches, which rely on predefined rules and struggle to adapt to dynamically changing environments in open-world settings. To overcome this, the authors propose CORE, a novel framework that enables online contextual safety reasoning without requiring prior maps or explicit safety specifications. CORE leverages a vision-language model (VLM) to interpret scene semantics in real time, grounds safety rules in physical space through spatial semantic grounding, and enforces safe control via control barrier functions (CBFs), while providing probabilistic safety guarantees that account for perception uncertainty. Experiments demonstrate that CORE significantly outperforms existing semantic safety methods in both simulation and real-world environments, effectively generating contextually appropriate safe behaviors even in previously unseen scenarios.

Technology Category

• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Robots are increasingly operating in open-world environments where safe behavior depends on context: the same hallway may require different navigation strategies when crowded versus empty, or during an emergency versus normal operations. Traditional safety approaches enforce fixed constraints in user-specified contexts, limiting their ability to handle the open-ended contextual variability of real-world deployment. We address this gap via CORE, a safety framework that enables online contextual reasoning, grounding, and enforcement without prior knowledge of the environment (e.g., maps or safety specifications). CORE uses a vision-language model (VLM) to continuously reason about context-dependent safety rules directly from visual observations, grounds these rules in the physical environment, and enforces the resulting spatially-defined safe sets via control barrier functions. We provide probabilistic safety guarantees for CORE that account for perceptual uncertainty, and we demonstrate through simulation and real-world experiments that CORE enforces contextually appropriate behavior in unseen environments, significantly outperforming prior semantic safety methods that lack online contextual reasoning. Ablation studies validate our theoretical guarantees and underscore the importance of both VLM-based reasoning and spatial grounding for enforcing contextual safety in novel settings. We provide additional resources at https://zacravichandran.github.io/CORE.

Problem

• contextual safety
• open-world robots
• contextual reasoning
• safety grounding
• dynamic environments

Innovation

• contextual safety reasoning
• vision-language model
• control barrier functions
• spatial grounding
• open-world robotics