Learning to Incentivize in Repeated Principal-Agent Problems with Adversarial Agent Arrivals

2025-05-29

AI Summary

This paper studies repeated incentive optimization for a principal facing adversarially arriving, multi-type agents with unknown types over a finite horizon, aiming to minimize cumulative regret relative to the optimal ex-post incentive policy. It introduces the first model of dynamic incentive learning under adversarial agent arrivals. Two practically feasible settings achieving sublinear regret are proposed: (i) known greedy response mapping, yielding a tight regret upper bound of $O(min{sqrt{KT log N}, Ksqrt{T}})$, and (ii) Lipschitz-continuous response, achieving $ ilde{O}((LN)^{1/3} T^{2/3})$ regret. Both bounds match the information-theoretic lower bounds up to logarithmic factors. Crucially, the framework supports selecting multiple incentive arms per round, enabling richer action spaces. The analysis integrates online learning, mechanism design, and adversarial robustness, providing foundational insights into incentive-compatible learning under worst-case agent behavior.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Adversarial Agents
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We initiate the study of a repeated principal-agent problem over a finite horizon $T$, where a principal sequentially interacts with $Kgeq 2$ types of agents arriving in an adversarial order. At each round, the principal strategically chooses one of the $N$ arms to incentivize for an arriving agent of unknown type. The agent then chooses an arm based on its own utility and the provided incentive, and the principal receives a corresponding reward. The objective is to minimize regret against the best incentive in hindsight. Without prior knowledge of agent behavior, we show that the problem becomes intractable, leading to linear regret. We analyze two key settings where sublinear regret is achievable. In the first setting, the principal knows the arm each agent type would select greedily for any given incentive. Under this setting, we propose an algorithm that achieves a regret bound of $O(min{sqrt{KTlog N},Ksqrt{T}})$ and provide a matching lower bound up to a $log K$ factor. In the second setting, an agent's response varies smoothly with the incentive and is governed by a Lipschitz constant $Lgeq 1$. Under this setting, we show that there is an algorithm with a regret bound of $ ilde{O}((LN)^{1/3}T^{2/3})$ and establish a matching lower bound up to logarithmic factors. Finally, we extend our algorithmic results for both settings by allowing the principal to incentivize multiple arms simultaneously in each round.

Problem

• Study repeated principal-agent problems with adversarial agent arrivals
• Minimize regret against best incentive without agent behavior knowledge
• Achieve sublinear regret in two key incentive settings

Innovation

• Adversarial agent arrivals handled strategically
• Sublinear regret with known greedy agent behavior
• Lipschitz smooth agent response leveraged algorithmically