What makes an Ensemble (Un) Interpretable?

2025-06-09

AI Summary

Ensemble models in machine learning—such as gradient-boosted trees—are widely deployed yet often treated as “black boxes,” with their interpretability lacking a rigorous mathematical foundation. Method: This paper introduces computational complexity theory—specifically the P ≠ NP assumption—into the analysis of ensemble interpretability. It formally models ensembles composed of decision trees and linear models, systematically characterizing how the number, size, and type of base learners affect the computational hardness of exact explanation. Contribution/Results: We establish the first strict complexity-theoretic classification framework for interpretability. Key results show that ensembles of constant-size decision trees admit polynomial-time exact explanations, whereas even ensembles comprising only a constant number of linear base models yield NP-hard explanation problems (unless P = NP). This work reveals a fundamental structural determinant of interpretability, bridging model architecture and computational tractability, and provides a foundational theoretical basis for explainable AI.

Technology Category

• Machine Learning: Ensemble Methods
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Ensemble models are widely recognized in the ML community for their limited interpretability. For instance, while a single decision tree is considered interpretable, ensembles of trees (e.g., boosted trees) are often treated as black-boxes. Despite this folklore recognition, there remains a lack of rigorous mathematical understanding of what particularly makes an ensemble (un)-interpretable, including how fundamental factors like the (1) *number*, (2) *size*, and (3) *type* of base models influence its interpretability. In this work, we seek to bridge this gap by applying concepts from computational complexity theory to study the challenges of generating explanations for various ensemble configurations. Our analysis uncovers nuanced complexity patterns influenced by various factors. For example, we demonstrate that under standard complexity assumptions like P$ eq$NP, interpreting ensembles remains intractable even when base models are of constant size. Surprisingly, the complexity changes drastically with the number of base models: small ensembles of decision trees are efficiently interpretable, whereas interpreting ensembles with even a constant number of linear models remains intractable. We believe that our findings provide a more robust foundation for understanding the interpretability of ensembles, emphasizing the benefits of examining it through a computational complexity lens.

Problem

• Understanding factors affecting ensemble model interpretability
• Analyzing impact of base model number, size, and type
• Exploring computational complexity of ensemble explanations

Innovation

• Applying computational complexity theory to ensembles
• Analyzing interpretability via base model factors
• Revealing tractability patterns in ensemble explanations