SafeBranch: Branch-Pair Safety Alignment for Embodied Agents

2026-08-20

AI Summary

为了解决基于视觉-语言模型的实体代理在执行任务时违反安全约束的问题，提出SafeBranch框架，通过环境回滚构建分支对来训练代理安全行为。

Abstract

Vision-language-model-based embodied agents can complete instructed tasks but often violate safety constraints in the process, a problem recently framed as interactive safety. Training such agents to act safely is difficult, since safety and task success are distinct objectives, and safety arises only at a small number of safety-critical steps within a trajectory. Standard supervision is insufficient: imitating safe trajectories teaches behavior without explaining why it is safe, and contrasting arbitrary safe and unsafe trajectories mixes the safety signal with unrelated differences. We propose SafeBranch, a framework that aligns an embodied actor on safety through branch pairs constructed from the actor's own unsafe rollouts via environment rollback. SafeBranch rolls each unsafe rollout back to the safety-critical step that caused the violation, queries the actor for a safe alternative, and pairs the original action with the alternative so that the two branches differ only at that step. The trained actor acts safely at deployment with no critic in the loop. On IS-Bench, SafetyALFRED, and out-of-distribution variants with unseen tasks and objects, it handles safety reliably without sacrificing task success, achieving roughly ten times more safe successes than the untrained baseline on the unseen-object variant.

Problem

• embodied agents
• interactive safety
• safety constraints
• task success
• safety-critical steps

Innovation

• branch pairs
• environment rollback
• safety-critical steps
• embodied agents