When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval

2026-06-15

AI Summary

Legal case retrieval remains highly challenging due to linguistic complexity and the need for precise lexical alignment, with BM25 still serving as a strong baseline. This work proposes a training-free, self-evolving framework that leverages large language model (LLM) agents within an automated evaluation environment to iteratively generate query rewriting rules, design validation experiments, and dynamically prune ineffective rules based on historical feedback, thereby optimizing BM25 performance. To the best of our knowledge, this is the first approach to endow rule-based query rewriting with self-evolution capabilities, effectively integrating LLMs’ prior knowledge with empirical experimental feedback. Evaluated on the LeCaRD-v2 Chinese legal retrieval benchmark, the method significantly outperforms non-evolutionary baselines—including handcrafted rules and greedy selection strategies—especially when powered by high-performance LLMs.

Abstract

Legal case retrieval remains challenging due to the complexity of legal language and the need for precise lexical alignment between queries and relevant cases. Although dense retrieval models have achieved notable progress, empirical studies show that BM25 continues to serve as a strong baseline in this domain. It motivates us to propose a self-evolving framework for rule-driven query rewriting that enhances BM25 without any parameter training. The framework equips an LLM-based agent with an automatic evaluation environment, enabling it to iteratively create rewriting rules, plan validation experiments over rule combinations, and eliminate ineffective rules based on historical feedbacks. We evaluate our method on the Chinese legal case retrieval benchmark LeCaRD-v2. Experimental results demonstrate that the proposed framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, particularly when powered by a highcapacity core LLM. We also conduct detailed analyses to investigate the mechanisms underlying self-evolution. Our findings reveal that LLM's capabilities to leverage previous experimental results and its intrinsic knowledge of rule elimination play critical roles in refining the rule set via self-evolution.

Problem

• legal case retrieval
• legal language complexity
• lexical alignment
• query rewriting
• information retrieval

Innovation

• self-evolving agent
• rule-driven query rewriting
• legal case retrieval
• BM25 enhancement
• LLM-based rule optimization