In Trust We Survive: Emergent Trust Learning

2026-03-18

AI Summary

In competitive multi-agent environments where agents rely solely on individual rewards and local observations, cooperation is vulnerable to resource depletion or exploitation. This work proposes ETL, a lightweight trust-driven control algorithm that seamlessly integrates into existing agents and dynamically modulates memory retention, exploration, and action selection through an internal trust state. Without requiring inter-agent communication or global information, ETL adaptively balances cooperative and competitive behaviors. Notably, it is the first approach to embed a compact trust mechanism directly into multi-agent decision-making with minimal computational overhead. Experimental results demonstrate that ETL significantly enhances cooperation rates and survival across diverse scenarios—including grid-based resource worlds, hierarchical tower tasks, and repeated prisoner’s dilemmas—effectively mitigating resource exhaustion and resisting sustained exploitation.

Technology Category

• Multiagent Systems: Mechanism Design
• Game Theory and Economic Paradigms: Cooperative Game Theory
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

We introduce Emergent Trust Learning (ETL), a lightweight, trust-based control algorithm that can be plugged into existing AI agents. It enables these to reach cooperation in competitive game environments under shared resources. Each agent maintains a compact internal trust state, which modulates memory, exploration, and action selection. ETL requires only individual rewards and local observations and incurs negligible computational and communication overhead. We evaluate ETL in three environments: In a grid-based resource world, trust-based agents reduce conflicts and prevent long-term resource depletion while achieving competitive individual returns. In a hierarchical Tower environment with strong social dilemmas and randomised floor assignments, ETL sustains high survival rates and recovers cooperation even after extended phases of enforced greed. In the Iterated Prisoner's Dilemma, the algorithm generalises to a strategic meta-game, maintaining cooperation with reciprocal opponents while avoiding long-term exploitation by defectors. Code will be released upon publication.

Problem

• trust
• cooperation
• competitive environments
• social dilemmas
• multi-agent systems

Innovation

• Emergent Trust Learning
• trust-based control
• multi-agent cooperation
• lightweight algorithm
• social dilemmas