Safe Task Planning with Long-Term Graph Memory for Embodied Agents

2026-09-08

AI Summary

为解决部分可观测环境下实体智能体任务规划中的安全问题，提出SafeMem框架，通过长期语义图记忆和基于大语言模型的风险预测来生成安全动作。

Abstract

Large language models (LLMs) and vision-language models (VLMs) have significantly advanced zero-shot task planning for embodied agents. However, most LLM- and VLM-driven methods struggle to generate safe high-level actions due to a lack of physical risk awareness, particularly under partial observability, where hazards lie outside the immediate field of view. To address this challenge, we propose a novel safe task-planning framework, SafeMem, which constructs and maintains a long-term semantic graph memory of the open and dynamic environment. Based on egocentric observations, the proposed framework incrementally accumulates knowledge about surrounding objects and their relationships with a graph. Then, an LLM-based risk predictor evaluates candidate actions using the graph memory, triggering a conservatism-modulated replanning loop with explanations for detected hazards. Extensive experiments on the IS-Bench benchmark and a real-world robot platform demonstrate that the SafeMem framework substantially improves safe success rates compared to state-of-the-art VLM-driven task planners. Video results are available on our webpage: https://sites.google.com/view/safemem.

Problem

• Safe Task Planning
• Physical Risk Awareness
• Partial Observability

Innovation

• long-term semantic graph memory
• risk awareness
• conservatism-modulated replanning