Robot Planning and Situation Handling with Active Perception
This work addresses the challenge of task execution failures in dynamic, open-world environments—such as those caused by jammed doors or unforeseen ground obstacles—by introducing the VAP-TAMP framework. VAP-TAMP uniquely integrates action-knowledge-guided active viewpoint selection with vision-language models and leverages scene graph construction and reasoning to enable joint task and motion planning (TAMP). The proposed approach facilitates real-time detection of and response to execution anomalies, significantly improving both task success rates and robotic autonomy in complex, dynamic settings. Evaluations on both simulated and real-world service robot platforms demonstrate its effectiveness in enhancing robustness and adaptability under uncertainty.