BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies

2025-05-31

AI Summary

To address the lack of interpretability and verifiability of deep reinforcement learning (DRL) policies in safety-critical applications, this paper proposes an online evolutionary framework that jointly optimizes behavioral diversity and policy simplicity via quality-diversity (QD) optimization and complexity-aware adaptive constraints. The method employs symbolic predicate logic to represent policies, enabling symbolic abstraction of state variables and online synthesis of compact, human-readable rule-based controllers. It introduces— for the first time—precise control over the number of logical rules and supports dynamic online evolution. Evaluated on CartPole, MountainCar, and Acrobot benchmarks, the generated policies average fewer than 15 rules while matching the performance of DRL baselines. This work achieves, for the first time, a competitive balance between strong interpretability and task performance in safety-critical RL settings.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The quest for interpretable reinforcement learning is a grand challenge for the deployment of autonomous decision-making systems in safety-critical applications. Modern deep reinforcement learning approaches, while powerful, tend to produce opaque policies that compromise verification, reduce transparency, and impede human oversight. To address this, we introduce BASIL (Best-Action Symbolic Interpretable Learning), a systematic approach for generating symbolic, rule-based policies via online evolutionary search with quality-diversity (QD) optimization. BASIL represents policies as ordered lists of symbolic predicates over state variables, ensuring full interpretability and tractable policy complexity. By using a QD archive, the methodology in the proposed study encourages behavioral and structural diversity between top-performing solutions, while a complexity-aware fitness encourages the synthesis of compact representations. The evolutionary system supports the use of exact constraints for rule count and system adaptability for balancing transparency with expressiveness. Empirical comparisons with three benchmark tasks CartPole-v1, MountainCar-v0, and Acrobot-v1 show that BASIL consistently synthesizes interpretable controllers with compact representations comparable to deep reinforcement learning baselines. Herein, this article introduces a new interpretable policy synthesis method that combines symbolic expressiveness, evolutionary diversity, and online learning through a unifying framework.

Problem

• Develops interpretable reinforcement learning policies for safety-critical applications
• Addresses opacity in deep RL policies via symbolic rule-based solutions
• Balances policy complexity and expressiveness using evolutionary diversity

Innovation

• Uses symbolic predicates for interpretable policies
• Employs quality-diversity evolutionary optimization
• Balances complexity with compact rule-based representations