ORCAID: Oblique Rule-Based Continuous-Action Interpretation for Deep RL Policies

2026-07-08

AI Summary

This work addresses the poor interpretability of deep reinforcement learning (RL) policies in continuous and hybrid action spaces by proposing a rule extraction method based on oblique decision trees. The approach partitions the state space using hyperplanes and fits local linear policies at leaf nodes. It employs a three-stage splitting procedure—comprising random initialization, local optimization, and backward elimination—along with leaf-node merging to efficiently construct compact, high-fidelity interpretable policies. This method achieves, for the first time, low-parameter, high-fidelity rule-based explanations of deep RL policies in continuous action spaces. Across multiple environments, it not only drastically reduces parameter count while maintaining strong performance but also enhances the original policy’s performance through knowledge distillation.

Abstract

Explainability remains a key issue in reinforcement learning (RL). Distilling an interpretable policy from an agent trained in a complex environment is particularly challenging when the action space is continuous. We introduce ORCAID, a novel method for extracting interpretable rule-based policies from RL agents operating in mixed continuous-discrete environments with continuous action spaces. Our main contribution is an efficient oblique decision tree training algorithm that partitions the state space by hyperplanes and fits local linear models. The key idea lies in a three-stage split search: efficient random initialization, local refinement, and backward elimination. Finally, adjacent leaves are merged to yield a concise set of interpretable rules describing a given deep RL policy. We evaluate ORCAID across multiple RL environments, demonstrating that the extracted rule-based policies maintain strong performance with a low number of parameters and can even be used to improve the performance of the original deep RL policy.

Problem

• Explainability
• Reinforcement Learning
• Continuous Action Space
• Interpretable Policy
• Rule Extraction

Innovation

• oblique decision tree
• rule extraction
• continuous action space
• interpretable reinforcement learning
• policy distillation