PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning

2026-08-29

AI Summary

PathBridger通过构建状态空间桥梁和使用逆动力学模型来解决离线目标条件强化学习中长时序任务的稀疏奖励传播问题。

Abstract

Offline goal-conditioned reinforcement learning (GCRL) aims to learn policies for reaching diverse goals entirely from fixed trajectory data. Long-horizon offline GCRL remains challenging because sparse goal-reaching signals must be propagated over many steps, while execution errors cannot be corrected through additional environment interaction. Existing methods address these challenges by improving long-range value estimation or reducing the effective decision horizon through subgoals, options, and action chunks. In several hierarchical methods, however, a selected subgoal specifies where to go, while the intervening state-space path remains implicit in an endpoint-conditioned low-level policy. To address this interface, we propose PathBridger, a hierarchical offline GCRL method that explicitly connects subgoal selection to short-horizon execution. PathBridger constructs a state-space bridge toward the selected intermediate endpoint and decodes it into a short executable action chunk using an inverse dynamics model. Experiments across the evaluated OGBench tasks demonstrate strong aggregate performance, with particularly large gains on the multi-object Cube manipulation tasks. Code: https://github.com/SChoish/PathBridger

Problem

• offline goal-conditioned reinforcement learning
• long-horizon
• sparse goal-reaching signals

Innovation

• hierarchical offline GCRL
• subgoal selection
• short-horizon execution
• state-space bridge
• inverse dynamics model