Lifted Forward Planning in Relational Factored Markov Decision Processes with Concurrent Actions

2025-05-28

AI Summary

To address the exponential explosion of state and action spaces in large-scale relational Markov decision processes (MDPs), exacerbated by increasing numbers of indistinguishable objects—especially under concurrent actions—this paper introduces the first efficient planning framework for first-order symbolic modeling. Methodologically: (i) it employs first-order logic and lifted inference to achieve polynomial-size symbolic compression of state and action spaces; (ii) it proposes Foreplan, the first relational forward planner, along with a fast approximate variant; and (iii) it supports automatic inference of the optimal number of objects required to complete a task. The framework is theoretically sound, with formal completeness guarantees. Empirical evaluation demonstrates speedups exceeding 10⁴× over conventional propositional and relational planners, significantly enhancing planning efficiency and scalability in domains with massive object counts.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Multiagent Systems: Multiagent Planning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Decision making is a central problem in AI that can be formalized using a Markov Decision Process. A problem is that, with increasing numbers of (indistinguishable) objects, the state space grows exponentially. To compute policies, the state space has to be enumerated. Even more possibilities have to be enumerated if the size of the action space depends on the size of the state space, especially if we allow concurrent actions. To tackle the exponential blow-up in the action and state space, we present a first-order representation to store the spaces in polynomial instead of exponential size in the number of objects and introduce Foreplan, a relational forward planner, which uses this representation to efficiently compute policies for numerous indistinguishable objects and actions. Additionally, we introduce an even faster approximate version of Foreplan. Moreover, Foreplan identifies how many objects an agent should act on to achieve a certain task given restrictions. Further, we provide a theoretical analysis and an empirical evaluation of Foreplan, demonstrating a speedup of at least four orders of magnitude.

Problem

• Exponential state space growth with indistinguishable objects
• Exponential action space complexity due to concurrent actions
• Efficient policy computation for numerous indistinguishable objects

Innovation

• First-order representation for polynomial space storage
• Relational forward planner for efficient policy computation
• Approximate version for faster performance