LePREC: Reasoning as Classification over Structured Factors for Assessing Relevance of Legal Issues

2026-04-21

AI Summary

This study addresses the global challenge that over half the world’s population lacks access to civil justice due to insufficient legal resources, compounded by the limited accuracy of current large language models (LLMs) in legal issue identification. To tackle this, the authors introduce a new dataset comprising 769 real Malaysian contract law cases and propose LePREC, a novel framework that first leverages LLMs such as GPT-4o to transform legal narratives into structured question-answer pairs, then applies a sparse linear model to weight discrete reasoning factors for classification. By integrating neural generation with symbolic reasoning, LePREC achieves a 30–40% improvement in relevance judgment accuracy over state-of-the-art baselines like GPT-4o and Claude, while maintaining data efficiency and significantly enhancing both precision and interpretability in legal issue identification.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

More than half of the global population struggles to meet their civil justice needs due to limited legal resources. While Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, significant challenges remain even at the foundational step of legal issue identification. To investigate LLMs' capabilities in this task, we constructed a dataset from 769 real-world Malaysian Contract Act court cases, using GPT-4o to extract facts and generate candidate legal issues, annotated by senior legal experts, which reveals a critical limitation: while LLMs generate diverse issue candidates, their precision remains inadequate (GPT-4o achieves only 62%). To address this gap, we propose LePREC (Legal Professional-inspired Reasoning Elicitation and Classification), a neuro-symbolic framework combining neural generation with structured statistical reasoning. LePREC consists of: (1) a neuro component leverages LLMs to transform legal descriptions into question-answer pairs representing diverse analytical factors, and (2) a symbolic component applies sparse linear models over these discrete features, learning explicit algebraic weights that identify the most informative reasoning factors. Unlike end-to-end neural approaches, LePREC achieves interpretability through transparent feature weighting while maintaining data efficiency through correlation-based statistical classification. Experiments show a 30-40% improvement over advanced LLM baselines, including GPT-4o and Claude, confirming that correlation-based factor-issue analysis offers a more data-efficient solution for relevance decisions.

Problem

• legal issue identification
• large language models
• relevance assessment
• civil justice
• legal reasoning

Innovation

• neuro-symbolic reasoning
• legal issue relevance
• structured factor classification
• interpretable LLMs
• sparse linear models