Learning from Imperfect Demonstrations via Temporal Behavior Tree-Guided Trajectory Repair

2026-04-05

AI Summary

This work addresses the challenge of learning robotic control policies from suboptimal, noisy, or imperfect demonstrations by proposing a trajectory refinement method based on Temporal Behavior Trees (TBTs). It introduces, for the first time, the use of TBTs to automatically correct demonstration trajectories that violate task specifications, thereby generating a logically consistent and interpretable dataset. Leveraging this refined data, the approach constructs a potential function to shape reward signals for reinforcement learning, enabling task-consistent policy learning without requiring an explicit model of system dynamics. Experimental results in grid-based navigation and continuous single- and multi-agent obstacle avoidance tasks demonstrate that the proposed method significantly improves both data efficiency and policy performance.

Abstract

Learning robot control policies from demonstrations is a powerful paradigm, yet real-world data is often suboptimal, noisy, or otherwise imperfect, posing significant challenges for imitation and reinforcement learning. In this work, we present a formal framework that leverages Temporal Behavior Trees (TBT), an extension of Signal Temporal Logic (STL) with Behavior Tree semantics, to repair suboptimal trajectories prior to their use in downstream policy learning. Given demonstrations that violate a TBT specification, a model-based repair algorithm corrects trajectory segments to satisfy the formal constraints, yielding a dataset that is both logically consistent and interpretable. The repaired trajectories are then used to extract potential functions that shape the reward signal for reinforcement learning, guiding the agent toward task-consistent regions of the state space without requiring knowledge of the agent's kinematic model. We demonstrate the effectiveness of this framework on discrete grid-world navigation and continuous single and multi-agent reach-avoid tasks, highlighting its potential for data-efficient robot learning in settings where high-quality demonstrations cannot be assumed.

Problem

• imperfect demonstrations
• trajectory repair
• robot learning
• imitation learning
• reinforcement learning

Innovation

• Temporal Behavior Trees
• trajectory repair
• imitation learning
• reward shaping
• formal specification