LLMPR: A Novel LLM-Driven Transfer Learning based Petition Ranking Model

2025-05-27

AI Summary

India’s judicial system suffers from severe case backlogs, and traditional manual prioritization of legal petitions is inefficient and highly subjective. This paper proposes the first automated petition prioritization framework integrating semantic representations—derived from legal-domain language models (e.g., LegalBERT)—with lightweight quantitative features (e.g., filing interval, character count). Crucially, we find that a simple model using only these numerical features achieves exceptional performance: R² = 0.988 and Spearman’s ρ = 0.998—matching or surpassing multimodal baselines (e.g., Random Forest and Decision Tree classifiers with >99% accuracy). This challenges the prevailing assumption that large language models are indispensable for such tasks. Our approach is rigorously validated on the ILDC dataset (7,593 manually annotated petitions), demonstrating substantial improvements in both timeliness and objectivity of judicial case triage.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Knowledge Representation and Reasoning: Preferences

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The persistent accumulation of unresolved legal cases, especially within the Indian judiciary, significantly hampers the timely delivery of justice. Manual methods of prioritizing petitions are often prone to inefficiencies and subjective biases further exacerbating delays. To address this issue, we propose LLMPR (Large Language Model-based Petition Ranking), an automated framework that utilizes transfer learning and machine learning to assign priority rankings to legal petitions based on their contextual urgency. Leveraging the ILDC dataset comprising 7,593 annotated petitions, we process unstructured legal text and extract features through various embedding techniques, including DistilBERT, LegalBERT, and MiniLM. These textual embeddings are combined with quantitative indicators such as gap days, rank scores, and word counts to train multiple machine learning models, including Random Forest, Decision Tree, XGBoost, LightGBM, and CatBoost. Our experiments demonstrate that Random Forest and Decision Tree models yield superior performance, with accuracy exceeding 99% and a Spearman rank correlation of 0.99. Notably, models using only numerical features achieve nearly optimal ranking results (R2 = 0.988, {ho} = 0.998), while LLM-based embeddings offer only marginal gains. These findings suggest that automated petition ranking can effectively streamline judicial workflows, reduce case backlog, and improve fairness in legal prioritization.

Problem

• Automating legal petition ranking to reduce judiciary backlog
• Addressing inefficiencies in manual prioritization of legal cases
• Improving fairness and speed in legal case processing

Innovation

• Uses transfer learning with LLM for legal ranking
• Combines textual and numerical features for training
• Achieves high accuracy with Random Forest models