Rule-Based Explanations for Retrieval-Augmented LLM Systems

2025-10-26

AI Summary

This work addresses the limited interpretability of retrieval-augmented large language models (RAG). We propose the first rule-based explanation framework for RAG, generating human-readable *if-then* rules that capture statistical associations between the presence or absence of retrieved evidence and model outputs. Methodologically, inspired by frequent itemset mining, we design an Apriori-style pruning strategy that integrates exhaustive pattern enumeration with efficient constraint-driven pruning, ensuring semantic plausibility while substantially improving computational efficiency. Experiments across multiple RAG benchmark tasks demonstrate that our approach rapidly produces high-fidelity, interpretable rules—achieving superior explanation quality and up to 60% reduction in rule discovery overhead compared to baselines. The framework establishes a lightweight, transparent, and verifiable attribution paradigm for RAG systems.

Technology Category

• Data Mining & Knowledge Management: Rule Mining & Pattern Mining
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

If-then rules are widely used to explain machine learning models; e.g., "if employed = no, then loan application = rejected." We present the first proposal to apply rules to explain the emerging class of large language models (LLMs) with retrieval-augmented generation (RAG). Since RAG enables LLM systems to incorporate retrieved information sources at inference time, rules linking the presence or absence of sources can explain output provenance; e.g., "if a Times Higher Education ranking article is retrieved, then the LLM ranks Oxford first." To generate such rules, a brute force approach would probe the LLM with all source combinations and check if the presence or absence of any sources leads to the same output. We propose optimizations to speed up rule generation, inspired by Apriori-like pruning from frequent itemset mining but redefined within the scope of our novel problem. We conclude with qualitative and quantitative experiments demonstrating our solutions' value and efficiency.

Problem

• Explain retrieval-augmented LLM systems using if-then rules
• Generate rules linking retrieved sources to LLM outputs
• Optimize rule generation with Apriori-inspired pruning techniques

Innovation

• Rule-based explanations for retrieval-augmented LLM systems
• Optimized rule generation using Apriori-inspired pruning
• Linking source presence/absence to explain output provenance