Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning

2026-07-10

AI Summary

This work addresses the high inference cost, complex training pipeline, and instability inherent in diffusion-based trajectory planning methods for offline reinforcement learning by proposing the Shortcut Trajectory Planning (STP) framework. STP employs a single-stage training procedure to learn a conditional shortcut trajectory model, eliminating the need for conventional teacher-student distillation. It enables efficient one-step or few-step inference with adjustable step sizes and incorporates a feasibility-aware critic mechanism to select high-quality candidate trajectories. Evaluated on multiple D4RL benchmark tasks, STP achieves strong performance while significantly simplifying the training process and enhancing both generation efficiency and stability.

Abstract

Diffusion-based trajectory planners have shown strong performance in offline reinforcement learning, but their iterative denoising process often incurs high inference cost. Consistency-based planners reduce the number of sampling steps, yet they typically rely on a two-stage teacher--student distillation pipeline that increases training cost and may introduce instability. We propose Shortcut Trajectory Planning (STP), an offline model-based reinforcement learning framework that incorporates shortcut models as efficient trajectory generators. STP trains a conditional shortcut trajectory model in a single stage, supports adjustable one-step and few-step inference through step-size conditioning, and selects candidate plans using a critic augmented with feasibility-aware correction. Across standard D4RL benchmarks, including locomotion, navigation, manipulation, and dexterous control tasks, STP achieves strong performance while simplifying the training pipeline for fast generative planning.

Problem

• offline reinforcement learning
• trajectory planning
• diffusion models
• inference efficiency
• training stability

Innovation

• shortcut trajectory planning
• offline reinforcement learning
• diffusion models
• step-size conditioning
• feasibility-aware critic