The Complexity of Sequential Prediction in Dynamical Systems

2024-02-09

arXiv.org

AI Summary

This paper addresses the problem of next-state prediction for model-free dynamical systems—i.e., predicting future states without knowledge of the underlying evolution function and without parametric assumptions. Methodologically, it introduces novel combinatorial metrics and dimensions to characterize the fundamental statistical complexity of model-free time-series prediction for the first time. Leveraging tools from combinatorial learning theory, dynamical systems analysis, and online learning regret analysis, the paper rigorously derives tight optimal mistake bounds (in the realizable setting) and regret bounds (in the agnostic setting). The results establish the first systematic theoretical foundation for model-free time-series prediction, precisely quantifying its intrinsic statistical difficulty and delineating fundamental algorithmic limits.

Technology Category

• Machine Learning: Other Foundations of Machine Learning
• Multiagent Systems: Other Foundations of Multi Agent Systems
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We study the problem of learning to predict the next state of a dynamical system when the underlying evolution function is unknown. Unlike previous work, we place no parametric assumptions on the dynamical system, and study the problem from a learning theory perspective. We define new combinatorial measures and dimensions and show that they quantify the optimal mistake and regret bounds in the realizable and agnostic setting respectively.

Problem

• Predicting next state in unknown dynamical systems without parametric assumptions
• Quantifying optimal mistake and regret bounds via new combinatorial measures
• Characterizing mistake growth and regret rates in realizable versus agnostic settings

Innovation

• No parametric assumptions on dynamical systems
• New combinatorial measures quantify bounds
• Mistakes grow with any increasing function