Laplace Transform Based Low-Complexity Learning of Continuous Markov Semigroups

2024-10-18

arXiv.org

AI Summary

Spectral estimation of the infinitesimal generator (IG) for continuous-time Markov processes is hindered by its unboundedness, while conventional methods lack theoretical recovery guarantees and suffer from high computational cost under small time lags. Method: We propose a data-driven spectral learning framework that deeply couples the IG resolvent with the Laplace transform—marking the first integration of resolvent analysis into Laplace-domain estimation. Contribution/Results: Our approach enables robust eigenvalue recovery even at small lags and provides the first provably unique spectral reconstruction guarantee. Theoretically, it subsumes existing transfer-operator-based methods; computationally, it reduces algorithmic complexity from O(n²) to O(n), greatly enhancing scalability. In two benchmark experiments, the method achieves high-accuracy IG spectral estimation under small time lags, demonstrating both effectiveness and numerical stability.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Markov processes serve as a universal model for many real-world random processes. This paper presents a data-driven approach for learning these models through the spectral decomposition of the infinitesimal generator (IG) of the Markov semigroup. The unbounded nature of IGs complicates traditional methods such as vector-valued regression and Hilbert-Schmidt operator analysis. Existing techniques, including physics-informed kernel regression, are computationally expensive and limited in scope, with no recovery guarantees for transfer operator methods when the time-lag is small. We propose a novel method that leverages the IG's resolvent, characterized by the Laplace transform of transfer operators. This approach is robust to time-lag variations, ensuring accurate eigenvalue learning even for small time-lags. Our statistical analysis applies to a broader class of Markov processes than current methods while reducing computational complexity from quadratic to linear in the state dimension. Finally, we illustrate the behaviour of our method in two experiments.

Problem

• Learning continuous Markov semigroups via spectral decomposition
• Overcoming unbounded infinitesimal generator complexity issues
• Ensuring robust eigenvalue learning for small time-lags

Innovation

• Spectral decomposition of infinitesimal generator
• Laplace transform of transfer operators
• Linear computational complexity in state dimension