Human-in-the-loop Learning Through Decentralized Communication Mechanisms

2025-09-11

AI Summary

Centralized information-sharing platforms (e.g., TripAdvisor, Waze) suffer from insufficient exploration due to selfish human behavior, degrading long-term performance in Human-in-the-Loop Learning (HILL). Method: We propose a decentralized communication mechanism that restricts global information flow and promotes local, asynchronous exploration to mitigate free-riding. This is the first work to introduce decentralized communication into human-in-the-loop learning, formalized via a multi-agent Markov decision process (MA-MDP) integrated with game-theoretic analysis. We design a distributed algorithm with linear time complexity that is asymptotically optimal. Contribution/Results: We theoretically prove that our mechanism strictly outperforms centralized baselines under standard assumptions. Empirical evaluation on real-world datasets demonstrates significant improvements in exploration efficiency and overall system learning performance.

Technology Category

• Multiagent Systems: Mechanism Design
• Humans and AI: Human-in-the-loop Machine Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Information sharing platforms like TripAdvisor and Waze involve human agents as both information producers and consumers. All these platforms operate in a centralized way to collect agents' latest observations of new options (e.g., restaurants, hotels, travel routes) and share such information with all in real time. However, after hearing the central platforms' live updates, many human agents are found selfish and unwilling to further explore unknown options for the benefit of others in the long run. To regulate the human-in-the-loop learning (HILL) game against selfish agents' free-riding, this paper proposes a paradigm shift from centralized to decentralized way of operation that forces agents' local explorations through restricting information sharing. When game theory meets distributed learning, we formulate our decentralized communication mechanism's design as a new multi-agent Markov decision process (MA-MDP), and derive its analytical condition to outperform today's centralized operation. As the optimal decentralized communication mechanism in MA-MDP is NP-hard to solve, we present an asymptotically optimal algorithm with linear complexity to determine the mechanism's timing of intermittent information sharing. Then we turn to non-myopic agents who may revert to even over-explore, and adapt our mechanism design to work. Simulation experiments using real-world dataset demonstrate the effectiveness of our decentralized mechanisms for various scenarios.

Problem

• Decentralized communication to regulate selfish human agents
• Overcoming free-riding in human-in-the-loop learning systems
• Designing mechanisms to force local exploration through restricted sharing

Innovation

• Decentralized communication mechanism design
• Multi-agent Markov decision process formulation
• Asymptotically optimal linear complexity algorithm