Towards Safe Robot Foundation Models

2025-03-10

AI Summary

Current research on robotic foundation models emphasizes task generalization but largely neglects safety guarantees required for real-world deployment. Generic policies often fail to satisfy safety constraints in dynamic physical environments. Method: We propose a plug-and-play universal safety layer that decouples pre-trained policies from safety enforcement via an architecture separating policy execution from constraint satisfaction. Our approach adapts the safety-critical reinforcement learning algorithm ATACOM to general-purpose robotic policies—without task-specific fine-tuning—by modeling a safe action space and applying real-time action projection. Contribution/Results: Evaluated on a physical air-hockey platform, our method eliminates collisions entirely while significantly improving safety and reliability during dynamic human-robot interaction. This marks the first successful integration of ATACOM with general robot policies, enabling cross-environment safety generalization without retraining or environment-specific adaptation.

Technology Category

• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Intelligent Robots: Other Foundations and Applications

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Security and privacy of machine learning and AI applications
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Robot foundation models hold the potential for deployment across diverse environments, from industrial applications to household tasks. While current research focuses primarily on the policies' generalization capabilities across a variety of tasks, it fails to address safety, a critical requirement for deployment on real-world systems. In this paper, we introduce a safety layer designed to constrain the action space of any generalist policy appropriately. Our approach uses ATACOM, a safe reinforcement learning algorithm that creates a safe action space and, therefore, ensures safe state transitions. By extending ATACOM to generalist policies, our method facilitates their deployment in safety-critical scenarios without requiring any specific safety fine-tuning. We demonstrate the effectiveness of this safety layer in an air hockey environment, where it prevents a puck-hitting agent from colliding with its surroundings, a failure observed in generalist policies.

Problem

• Ensuring robot foundation models operate safely in diverse environments
• Addressing lack of safety in generalist policies for real-world deployment
• Preventing collisions in safety-critical scenarios using a safety layer

Innovation

• Safety layer constrains action space
• Extends ATACOM to generalist policies
• Ensures safe state transitions