On Piecewise Affine Reachability with Bellman Operators

2025-02-27

AI Summary

This paper investigates the reachability problem for piecewise-affine maps induced by the Bellman operator of Markov decision processes: given an initial vector, a target vector, and such a map, determine whether there exists a natural number (n) such that the (n)-fold iteration reaches the target. Although reachability is known to be undecidable for general two-dimensional piecewise-affine systems, we establish, for the first time, its decidability for Bellman-type piecewise-affine maps in two dimensions—and extend this result to arbitrary finite dimensions. Our approach integrates dynamic programming principles (monotonicity and contraction), computational geometry (linear partitioning structure), and mathematical logic to construct a decision framework based on geometric region decomposition and precise characterization of iterative behavior. This work breaks through a classical undecidability barrier and provides the first decidable symbolic verification framework for controlled dynamical systems in high dimensions.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Security and Privacy: Large-scale security measurements

Abstract

A piecewise affine map is one of the simplest mathematical objects exhibiting complex dynamics. The reachability problem of piecewise affine maps is given as follows: Given two vectors $mathbf{s}, mathbf{t} in mathbb{Q}^d$ and a piecewise affine map $f$, is there $nin mathbb{N}$ such that $f^{n}(mathbf{s}) = mathbf{t}$? Koiran, Cosnard, and Garzon show that the reachability problem of piecewise affine maps is undecidable even in dimension 2. Most of the recent progress has been focused on decision procedures for one-dimensional piecewise affine maps, where the reachability problem has been shown to be decidable for some subclasses. However, the general undecidability discouraged research into positive results in arbitrary dimension. In this work, we consider a rich subclass of piecewise affine maps defined by Bellman operators of Markov decision processes (MDPs). We then investigate the restriction of the piecewise affine reachability problem to that with Bellman operators and, in particular, its decidability in any dimension. As one of our primary contributions, we establish the decidability of reachability for two-dimensional Bellman operators, in contrast to the negative result known for general piecewise affine maps.

Problem

• Decidability of reachability in piecewise affine maps
• Focus on Bellman operators in Markov decision processes
• Establish decidability for two-dimensional Bellman operators

Innovation

• Bellman operators in MDPs
• Decidability in 2D analysis
• Restricted piecewise affine maps