Markovian Pandora's box

2025-01-31

AI Summary

This paper studies the Pandora’s Box problem under Markovian dependencies: sequential box inspections are modeled over a shared directed acyclic graph (DAG), where box rewards evolve as a Markov chain and inspection order is constrained by the DAG structure. It is the first work to incorporate Markovian reward dependencies into this classical model, proposing an algorithmic framework for cost-sensitive sequential decision-making. Theoretical contributions include: (1) an optimal fully adaptive policy for forest-structured DAGs, along with an efficient algorithm achieving a 1/2 approximation ratio in near-linear time; (2) near-optimal expected reward for multi-chain DAGs. Both algorithms exhibit significantly lower computational complexity than exact methods, with accelerating scalability advantages as the graph size grows—providing scalable theoretical foundations and practical algorithms for sequential exploration in data- and computation-driven settings.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Reasoning under Uncertainty: Sequential Decision Making
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

In this paper, we study the Markovian Pandora's Box Problem, where decisions are governed by both order constraints and Markovianly correlated rewards, structured within a shared directed acyclic graph. To the best of our knowledge, previous work has not incorporated Markovian dependencies in this setting. This framework is particularly relevant to applications such as data or computation driven algorithm design, where exploration of future models incurs cost. We present optimal fully adaptive strategies where the associated graph forms a forest. Under static transition, we introduce a strategy that achieves a near optimal expected payoff in multi line graphs and a 1/2 approximation in forest-structured graphs. Notably, this algorithm provides a significant speedup over the exact solution, with the improvement becoming more pronounced as the graph size increases. Our findings deepen the understanding of sequential exploration under Markovian correlations in graph-based decision-making.

Problem

• Markovian Pandora's Box Problem
• Complex Graph Structures
• Optimization of Algorithm Design

Innovation

• Markovian Pandora's Box Problem
• Decision-making Algorithm
• Computational Efficiency