Multi-criteria Rank-based Aggregation for Explainable AI

2025-05-30

AI Summary

Existing XAI methods often yield conflicting explanations for the same prediction, and balancing multidimensional quality criteria—such as complexity, fidelity, and stability—remains challenging. Method: This paper proposes a weighted ranking aggregation framework grounded in Multi-Criteria Decision Making (MCDM), pioneering the integration of TOPSIS and the Weighted Sum Method (WSUM) into XAI explanation fusion. It introduces a novel, rank-based paradigm for quantifying XAI quality, circumventing biases inherent in scalar normalization. Built upon black-box explainers (e.g., LIME, SHAP), the framework enables robust cross-method explanation aggregation. Contribution/Results: Experiments across multiple public datasets demonstrate that the method significantly improves the trade-off between explanation stability and fidelity. It outperforms individual explainers and state-of-the-art aggregation approaches in comprehensive evaluation, exhibiting superior robustness under multi-metric assessment.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Machine Learning: Learning Preferences or Rankings
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Explainability is crucial for improving the transparency of black-box machine learning models. With the advancement of explanation methods such as LIME and SHAP, various XAI performance metrics have been developed to evaluate the quality of explanations. However, different explainers can provide contrasting explanations for the same prediction, introducing trade-offs across conflicting quality metrics. Although available aggregation approaches improve robustness, reducing explanations' variability, very limited research employed a multi-criteria decision-making approach. To address this gap, this paper introduces a multi-criteria rank-based weighted aggregation method that balances multiple quality metrics simultaneously to produce an ensemble of explanation models. Furthermore, we propose rank-based versions of existing XAI metrics (complexity, faithfulness and stability) to better evaluate ranked feature importance explanations. Extensive experiments on publicly available datasets demonstrate the robustness of the proposed model across these metrics. Comparative analyses of various multi-criteria decision-making and rank aggregation algorithms showed that TOPSIS and WSUM are the best candidates for this use case.

Problem

• Balancing multiple XAI quality metrics for robust explanations
• Reducing variability in explanations from conflicting metrics
• Evaluating ranked feature importance with new rank-based metrics

Innovation

• Multi-criteria rank-based weighted aggregation method
• Rank-based XAI metrics for evaluation
• TOPSIS and WSUM as optimal algorithms