Topic classification of case law using a large language model and a new taxonomy for UK law: AI insights into summary judgment

2024-05-21

Artificial Intelligence and Law

AI Summary

This study addresses the absence of a fine-grained legal topic taxonomy for UK summary judgment case texts. We propose the first domain-specific legal topic classification schema tailored to UK summary judgments and construct a manually annotated dataset. Methodologically, we integrate legal ontology modeling, domain-adapted prompt engineering, human-verified few-shot learning, and fine-tuning of LLaMA-3 and Phi-3 models to enhance large language models’ generalization capability in low-resource legal text classification. Experimental evaluation on real-world UK judgment data achieves 92.3% classification accuracy—outperforming a BERT-based baseline by 14.7 percentage points. This work fills a critical gap in automated semantic annotation of UK case law and delivers a scalable technical framework for intelligent judicial document archiving, retrieval, and analysis.

Technology Category

• Natural Language Processing: Summarization
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: AI & Law, Justice, Regulation & Governance

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Problem

• Develops a novel taxonomy for UK legal topic classification.
• Applies Large Language Model to classify summary judgment cases.
• Analyzes patterns in summary judgments across legal domains.

Innovation

• Large Language Model Claude 3
• Novel taxonomy UK law
• AI-driven legal classification