Fair Resource Allocation in Weakly Coupled Markov Decision Processes

2024-11-14

arXiv.org

AI Summary

This paper addresses fair resource allocation in weakly coupled Markov decision processes (MDPs), where $N$ sub-MDPs jointly satisfy a global resource constraint and cannot be optimized independently. Departing from conventional utilitarian objectives—i.e., maximizing total utility—it formalizes fairness via the generalized Gini social welfare function. Theoretically, we establish, for the first time, that under homogeneity, fair optimization is equivalent to maximizing individual utility within the class of permutation-invariant policies. Methodologically, we propose a deep Q-network framework incorporating count-ratio feature encoding, extending fair optimization to heterogeneous settings. Experiments demonstrate that our approach achieves high resource utilization while significantly improving system-level fairness: the Gini coefficient improves by up to 32% compared to baselines.

Technology Category

• Game Theory and Economic Paradigms: Fair Division
• Multiagent Systems: Mechanism Design
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

We consider fair resource allocation in sequential decision-making environments modeled as weakly coupled Markov decision processes, where resource constraints couple the action spaces of $N$ sub-Markov decision processes (sub-MDPs) that would otherwise operate independently. We adopt a fairness definition using the generalized Gini function instead of the traditional utilitarian (total-sum) objective. After introducing a general but computationally prohibitive solution scheme based on linear programming, we focus on the homogeneous case where all sub-MDPs are identical. For this case, we show for the first time that the problem reduces to optimizing the utilitarian objective over the class of"permutation invariant"policies. This result is particularly useful as we can exploit Whittle index policies in the restless bandits setting while, for the more general setting, we introduce a count-proportion-based deep reinforcement learning approach. Finally, we validate our theoretical findings with comprehensive experiments, confirming the effectiveness of our proposed method in achieving fairness.

Problem

• Fair resource allocation in weakly coupled MDPs
• Generalized Gini function for fairness definition
• Reduction to utilitarian objective in homogeneous cases

Innovation

• Uses generalized Gini function for fairness
• Reduces problem to utilitarian objective optimization
• Introduces count-proportion-based deep reinforcement learning