Don't Let Your Robot be Harmful: Responsible Robotic Manipulation via Safety-as-Policy

2024-11-27

AI Summary

Robots executing instructions without safety awareness pose tangible risks—including poisoning and fire hazards. To address this, we propose the “Safety-as-Policy” paradigm, which formalizes safety as a learnable decision policy: a generative world model simulates hazardous scenarios, while a reflective mental model enables safety-aware cognition and causal reasoning. We introduce SafeBox, the first responsibility-aware manipulation benchmark for evaluating safety-critical robotic manipulation. Our method integrates virtual interactive training, synthetic data generation, and co-optimization of the world and mental models. Evaluated on SafeBox and real-world experiments, our approach achieves a 32% improvement in task success rate and a 96.5% risk avoidance rate—substantially outperforming baselines—and demonstrates effective transfer of safety knowledge from simulation to physical deployment.

Technology Category

• Intelligent Robots: Manipulation
• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Unthinking execution of human instructions in robotic manipulation can lead to severe safety risks, such as poisonings, fires, and even explosions. In this paper, we present responsible robotic manipulation, which requires robots to consider potential hazards in the real-world environment while completing instructions and performing complex operations safely and efficiently. However, such scenarios in real world are variable and risky for training. To address this challenge, we propose Safety-as-policy, which includes (i) a world model to automatically generate scenarios containing safety risks and conduct virtual interactions, and (ii) a mental model to infer consequences with reflections and gradually develop the cognition of safety, allowing robots to accomplish tasks while avoiding dangers. Additionally, we create the SafeBox synthetic dataset, which includes one hundred responsible robotic manipulation tasks with different safety risk scenarios and instructions, effectively reducing the risks associated with real-world experiments. Experiments demonstrate that Safety-as-policy can avoid risks and efficiently complete tasks in both synthetic dataset and real-world experiments, significantly outperforming baseline methods. Our SafeBox dataset shows consistent evaluation results with real-world scenarios, serving as a safe and effective benchmark for future research.

Problem

• Prevent robots from causing safety risks during manipulation tasks
• Develop models to simulate and assess real-world hazardous scenarios
• Create a synthetic dataset for safe robotic training and evaluation

Innovation

• Safety-as-policy integrates world and mental models
• World model generates risky scenarios for training
• Mental model infers consequences to ensure safety