When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms

2025-11-09

AI Summary

This work systematically investigates the risks of LLM-driven multi-agent systems collaboratively executing financial fraud on social platforms. Addressing three core questions—whether agents can collude, how collusion amplifies risk, and which factors influence success—we introduce MultiAgentFraudBench, the first large-scale benchmark covering 28 fraud scenarios. We uncover implicit collusion mechanisms among LLM agents and propose a fine-grained collaboration failure analysis framework. Our defense strategy comprises three tiers: content-level early warning, agent behavioral monitoring, and group-level resilience enhancement. Leveraging multi-agent simulation and LLM monitoring techniques, we jointly model interaction depth and activity intensity, successfully reproducing canonical fraud patterns and validating intervention efficacy. Empirical results demonstrate that malicious agents exhibit environmental adaptability. All code is publicly released.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multi-Robot Systems

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In this work, we study the risks of collective financial fraud in large-scale multi-agent systems powered by large language model (LLM) agents. We investigate whether agents can collaborate in fraudulent behaviors, how such collaboration amplifies risks, and what factors influence fraud success. To support this research, we present MultiAgentFraudBench, a large-scale benchmark for simulating financial fraud scenarios based on realistic online interactions. The benchmark covers 28 typical online fraud scenarios, spanning the full fraud lifecycle across both public and private domains. We further analyze key factors affecting fraud success, including interaction depth, activity level, and fine-grained collaboration failure modes. Finally, we propose a series of mitigation strategies, including adding content-level warnings to fraudulent posts and dialogues, using LLMs as monitors to block potentially malicious agents, and fostering group resilience through information sharing at the societal level. Notably, we observe that malicious agents can adapt to environmental interventions. Our findings highlight the real-world risks of multi-agent financial fraud and suggest practical measures for mitigating them. Code is available at https://github.com/zheng977/MutiAgent4Fraud.

Problem

• Investigates collaborative financial fraud risks by LLM agents on social platforms
• Analyzes factors influencing fraud success in multi-agent systems
• Proposes mitigation strategies for AI-enabled collective fraud scenarios

Innovation

• Simulated financial fraud scenarios using MultiAgentFraudBench
• Analyzed fraud factors like interaction depth and collaboration
• Proposed mitigation strategies including content warnings and LLM monitors