(Sometimes) Less is More: Mitigating the Complexity of Rule-based Representation for Interpretable Classification

2025-09-26

AI Summary

Deep learning models often fail to meet high-transparency requirements due to their “black-box” nature. To address the excessive complexity of Concept Rule Sets (CRS)—a class of interpretable models—where accuracy, rule conciseness, and interpretability are difficult to balance, this paper proposes an end-to-end differentiable L0-regularized sparsification method embedded within the Multi-Layer Logic Perceptron (MLLP) framework. Unlike heuristic sparsification strategies (e.g., stochastic binarization), our approach introduces a differentiable L0 norm into the loss function, enabling continuous optimization and automatic pruning of logical weights. This simultaneously reduces both the number and length of CRS rules. Experiments demonstrate that the method preserves high classification accuracy while substantially compressing rule complexity, achieving a superior trade-off between interpretability and performance. The proposed framework provides a novel pathway toward compact, trustworthy, and logically interpretable models.

Technology Category

• Machine Learning: Transparent, Interpretable, Explainable ML
• Computer Vision: Interpretability, Explainability, and Transparency
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Deep neural networks are widely used in practical applications of AI, however, their inner structure and complexity made them generally not easily interpretable. Model transparency and interpretability are key requirements for multiple scenarios where high performance is not enough to adopt the proposed solution. In this work, a differentiable approximation of $L_0$ regularization is adapted into a logic-based neural network, the Multi-layer Logical Perceptron (MLLP), to study its efficacy in reducing the complexity of its discrete interpretable version, the Concept Rule Set (CRS), while retaining its performance. The results are compared to alternative heuristics like Random Binarization of the network weights, to determine if better results can be achieved when using a less-noisy technique that sparsifies the network based on the loss function instead of a random distribution. The trade-off between the CRS complexity and its performance is discussed.

Problem

• Reducing complexity of interpretable rule-based classifiers
• Balancing model performance with interpretability requirements
• Comparing differentiable regularization against heuristic sparsification methods

Innovation

• Differentiable L0 regularization for logic networks
• Sparsifying network based on loss function
• Reducing complexity while retaining performance