Guided Riemannian Optimization (GuRO): Bridging Model Predictive Control and Decision Transformers

2026-08-24

AI Summary

本文提出一种结合模型预测控制和强化学习的序列决策框架，通过黎曼优化方法解决高维非线性系统的决策问题，提高了优化效率和稳定性。

Abstract

Decision-making in high-dimensional, nonlinear systems remains a central challenge in robotics. While model-based methods like Model Predictive Control (MPC) offer sample efficiency and interpretability, their performance degrades when the dynamics model is inaccurate or long-horizon predictions are required. Conversely, model-free reinforcement learning (RL) learns policies directly from interaction but suffers from high sample complexity and unstable optimization. Recent advances in sequence modeling have inspired transformer-based decision-making frameworks that can unify MPC and RL, but their training typically faces significant optimization challenges due to highly non-convex loss landscapes. In this work, we propose a novel framework that integrates MPC with RL in a sequence decision-making framework and leverages a curvature-aware optimization to efficiently tackle non-convex loss landscapes. MPC provides predictions of locally optimal trajectories that guide the decision transformer, removing the need for extensive offline pretraining. To address the slow and unstable convergence of traditional optimizers, we train the policy in a Riemannian parameter space using an efficient Riemannian (curvature-aware) method, leading to faster and more robust optimization. We evaluate our framework on high-dimensional quadruped control tasks and demonstrate consistent improvements over strong baselines, including TRPO, SAC, and Online Decision Transformer, achieving higher returns and faster convergence.

Problem

• high-dimensional
• nonlinear systems
• decision-making
• optimization challenges
• non-convex loss landscapes

Innovation

• Guided Riemannian Optimization
• Model Predictive Control
• Decision Transformers
• Curvature-aware Optimization
• Riemannian Parameter Space