Large Language Models Meet Legal Artificial Intelligence: A Survey

2025-09-12

AI Summary

This paper addresses the fragmented state of research and inconsistent evaluation practices in legal artificial intelligence (Legal AI) powered by large language models (LLMs). To tackle these challenges, we propose the first comprehensive technical landscape: systematically cataloging 16 LLM families, 47 task-specific frameworks, 15 benchmark suites, and 29 domain-specific datasets; introducing a multidimensional evaluation framework covering legal understanding, reasoning, and generation; and open-sourcing an integrated resource platform. Our analysis identifies persistent limitations across current models—including domain expertise, interpretability, and regulatory compliance—while offering Legal AI researchers a structured entry point and reusable infrastructure. The work significantly enhances the systematicity, reproducibility, and cross-study comparability of Legal AI research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Philosophy and Ethics of AI: AI & Law, Justice, Regulation & Governance

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) have significantly advanced the development of Legal Artificial Intelligence (Legal AI) in recent years, enhancing the efficiency and accuracy of legal tasks. To advance research and applications of LLM-based approaches in legal domain, this paper provides a comprehensive review of 16 legal LLMs series and 47 LLM-based frameworks for legal tasks, and also gather 15 benchmarks and 29 datasets to evaluate different legal capabilities. Additionally, we analyse the challenges and discuss future directions for LLM-based approaches in the legal domain. We hope this paper provides a systematic introduction for beginners and encourages future research in this field. Resources are available at https://github.com/ZhitianHou/LLMs4LegalAI.

Problem

• Surveying LLM-based approaches for Legal AI development
• Reviewing legal LLMs, frameworks, benchmarks, and datasets
• Analyzing challenges and future directions for legal LLMs

Innovation

• Reviewing 16 legal LLM series
• Analyzing 47 LLM-based legal frameworks
• Gathering 15 benchmarks and 29 datasets