Explaining k-Nearest Neighbors: Abductive and Counterfactual Explanations

2025-01-10

AI Summary

High-dimensional k-nearest neighbors (k-NN) classifiers lack interpretability, hindering trust and deployment in safety-critical applications. Method: We formalize and compute two fundamental explanation types—minimal sufficient reasons (abductive explanations) and minimal perturbation counterfactuals—from a feature-centric perspective. We propose efficient algorithms based on integer quadratic programming (IQP) and SAT encoding to generate exact explanations, even for NP-hard variants. Contribution/Results: This work establishes the first systematic computational complexity characterization of both explanation types over discrete and continuous feature spaces, revealing the decisive role of distance metrics in explainability. Despite inherent hardness, our algorithms scale to high-dimensional real-world datasets while guaranteeing correctness. Experiments demonstrate feasibility and accuracy across benchmark datasets, advancing both the theoretical understanding and practical applicability of k-NN interpretability.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Transparent, Interpretable, Explainable ML
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Despite the wide use of $k$-Nearest Neighbors as classification models, their explainability properties remain poorly understood from a theoretical perspective. While nearest neighbors classifiers offer interpretability from a"data perspective", in which the classification of an input vector $ar{x}$ is explained by identifying the vectors $ar{v}_1, ldots, ar{v}_k$ in the training set that determine the classification of $ar{x}$, we argue that such explanations can be impractical in high-dimensional applications, where each vector has hundreds or thousands of features and it is not clear what their relative importance is. Hence, we focus on understanding nearest neighbor classifications through a"feature perspective", in which the goal is to identify how the values of the features in $ar{x}$ affect its classification. Concretely, we study abductive explanations such as"minimum sufficient reasons", which correspond to sets of features in $ar{x}$ that are enough to guarantee its classification, and"counterfactual explanations"based on the minimum distance feature changes one would have to perform in $ar{x}$ to change its classification. We present a detailed landscape of positive and negative complexity results for counterfactual and abductive explanations, distinguishing between discrete and continuous feature spaces, and considering the impact of the choice of distance function involved. Finally, we show that despite some negative complexity results, Integer Quadratic Programming and SAT solving allow for computing explanations in practice.

Problem

• High-dimensional Feature Space
• K-Nearest Neighbors Algorithm
• Feature Importance

Innovation

• K-Nearest Neighbors
• Feature Selection
• Integer Quadratic Programming