Learning with Imperfect Models: When Multi-step Prediction Mitigates Compounding Error

2025-04-02

AI Summary

In learning-based control, single-step autoregressive prediction suffers from error accumulation in closed-loop operation, leading to performance degradation—especially under model misspecification (e.g., partial observability). This work focuses on linear dynamical systems and provides the first rigorous characterization of the bias–complexity trade-off between single-step and multi-step prediction. We theoretically establish that multi-step prediction significantly reduces asymptotic bias even under imperfect models, and its benefits outweigh the associated increase in modeling complexity. Methodologically, we propose training single-step models using a multi-step loss, complemented by closed-loop performance evaluation and tight error bound analysis. Numerical experiments demonstrate that our approach outperforms standard single-step methods both in open-loop prediction accuracy and closed-loop robustness. This work furnishes both theoretical foundations and a practical design paradigm for learning-based controllers.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Multiagent Systems: Multiagent Learning
• Machine Learning: Online Learning & Bandits

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Compounding error, where small prediction mistakes accumulate over time, presents a major challenge in learning-based control. For example, this issue often limits the performance of model-based reinforcement learning and imitation learning. One common approach to mitigate compounding error is to train multi-step predictors directly, rather than relying on autoregressive rollout of a single-step model. However, it is not well understood when the benefits of multi-step prediction outweigh the added complexity of learning a more complicated model. In this work, we provide a rigorous analysis of this trade-off in the context of linear dynamical systems. We show that when the model class is well-specified and accurately captures the system dynamics, single-step models achieve lower asymptotic prediction error. On the other hand, when the model class is misspecified due to partial observability, direct multi-step predictors can significantly reduce bias and thus outperform single-step approaches. These theoretical results are supported by numerical experiments, wherein we also (a) empirically evaluate an intermediate strategy which trains a single-step model using a multi-step loss and (b) evaluate performance of single step and multi-step predictors in a closed loop control setting.

Problem

• Analyzes compounding error in learning-based control systems
• Compares single-step vs multi-step prediction model performance
• Evaluates model misspecification impact on prediction accuracy

Innovation

• Multi-step predictors reduce compounding error
• Analyze trade-off in linear dynamical systems
• Multi-step loss improves single-step models