Evidence-Gated Task and Motion Planning with Vision-Language Models

2026-08-20

AI Summary

本文提出EAFG框架，通过视觉证据获取和可行性判断解决机器人在执行长时操作任务中因部分可观测性导致的不确定性问题。

Abstract

Robots executing long-horizon manipulation tasks from natural-language instructions must reason about both semantic task structure and geometric feasibility. However, under partial observability, the availability of goal-relevant objects may be uncertain. In such cases, approaches that combine Vision-Language Models (VLMs) with Task and Motion Planning (TAMP) may generate subgoals that rely on the VLM's prior knowledge without observational support, leading to execution failures or unintended outcomes. We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework that acquires visual evidence through VLM-generated exploratory subgoals and TAMP-based execution. EAFG then applies a feasibility gate to decide whether to proceed with task planning, acquire further evidence, or halt. Our experiments show that, in cooking tasks with ambiguous object use, EAFG improves recipe completion by discovering task-relevant objects before planning. For instructions requiring an absent object, EAFG promotes appropriate halt decisions and reduces repeated attempts to manipulate that object.

Problem

• Robots
• natural-language instructions
• partial observability
• Vision-Language Models (VLMs)
• Task and Motion Planning (TAMP)

Innovation

• Evidence Acquisition and Feasibility Gating (EAFG)
• Vision-Language Models (VLMs)
• Task and Motion Planning (TAMP)