AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents

2024-08-15

arXiv.org

AI Summary

Existing legal large language models (LLMs) fail to authentically simulate the dynamic, adversarial nature of real courtroom proceedings. To address this, we propose an LLM-based autonomous legal agent system that introduces the first end-to-end courtroom process simulation framework. Our method leverages multi-agent collaboration and adversarial evolutionary mechanisms, enabling plaintiff and defendant agents to iteratively learn from authentic judicial precedents across thousands of simulated trials. We integrate a structured legal knowledge graph with verifiable prompt engineering to ensure reproducible, domain-accurate reasoning. Experimental evaluation demonstrates significant improvements in response latency, legal argument rigor, and practical logical coherence; these gains are empirically validated by a panel of licensed practicing attorneys. This work establishes the first simulation paradigm for legal AI endowed with dynamic evolutionary capability and high judicial-practice fidelity.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

In this paper, we present a simulation system called AgentCourt that simulates the entire courtroom process. The judge, plaintiff's lawyer, defense lawyer, and other participants are autonomous agents driven by large language models (LLMs). Our core goal is to enable lawyer agents to learn how to argue a case, as well as improving their overall legal skills, through courtroom process simulation. To achieve this goal, we propose an adversarial evolutionary approach for the lawyer-agent. Since AgentCourt can simulate the occurrence and development of court hearings based on a knowledge base and LLM, the lawyer agents can continuously learn and accumulate experience from real court cases. The simulation experiments show that after two lawyer-agents have engaged in a thousand adversarial legal cases in AgentCourt (which can take a decade for real-world lawyers), compared to their pre-evolutionary state, the evolved lawyer agents exhibit consistent improvement in their ability to handle legal tasks. To enhance the credibility of our experimental results, we enlisted a panel of professional lawyers to evaluate our simulations. The evaluation indicates that the evolved lawyer agents exhibit notable advancements in responsiveness, as well as expertise and logical rigor. This work paves the way for advancing LLM-driven agent technology in legal scenarios. Code is available at https://github.com/relic-yuexi/AgentCourt.

Problem

• Modeling dynamic courtroom interactions with LLM-based agents
• Overcoming static knowledge limitations in legal language models
• Enhancing legal reasoning through adversarial evolutionary learning

Innovation

• Adversarial evolutionary approach for agent learning
• Dynamic knowledge learning in courtroom simulations
• Evolving legal knowledge base enhances reasoning