AIMing for Standardised Explainability Evaluation in GNNs: A Framework and Case Study on Graph Kernel Networks

2026-05-14

AI Summary

This study addresses the lack of a unified and general evaluation framework for explainability in graph neural networks (GNNs), which hinders cross-model comparisons. To bridge this gap, the authors propose the AIM evaluation framework, which systematically assesses GNN explainability along three dimensions: accuracy, instance-level fidelity, and model-level interpretability, thereby establishing the first general-purpose benchmark applicable across diverse GNN architectures. Applying AIM to intrinsically interpretable models such as Graph Kernel Networks (GKNs) reveals inherent limitations in their explanations. Motivated by these insights, the authors design an enhanced model, xGKN, which significantly improves explainability while maintaining high predictive performance. This work not only introduces a novel paradigm for evaluating GNN explainability but also demonstrates how principled evaluation can directly inform and drive model improvement.

Abstract

Graph Neural Networks (GNNs) have advanced significantly in handling graph-structured data, but a comprehensive framework for evaluating explainability remains lacking. Existing evaluation frameworks primarily involve post-hoc explanations, and operate in the setting where multiple methods generate a suite of explanations for a single model. This makes comparison of explanations across models difficult. Evaluation of inherently interpretable models often targets a specific aspect of interpretability relevant to the model, but remains underdeveloped in terms of generating insight across a suite of measures. We introduce AIM, a comprehensive framework that addresses these limitations by measuring Accuracy, Instance-level explanations, and Model-level explanations. AIM is formulated with minimal constraints to enhance flexibility and facilitate broad applicability. Here, we use AIM in a pipeline, extracting explanations from inherently interpretable GNNs such as graph kernel networks (GKNs) and prototype networks (PNs), evaluating these explanations with AIM, identifying their limitations and obtaining insights to their characteristics. Taking GKNs as a case study, we show how the insights obtained from AIM can be used to develop an updated model, xGKN, that maintains high accuracy while demonstrating improved explainability. Our approach aims to advance the field of Explainable AI (XAI) for GNNs, providing more robust and practical solutions for understanding and improving complex models.

Problem

• Explainability Evaluation
• Graph Neural Networks
• Interpretable Models
• Standardized Framework
• Model Comparison

Innovation

• Explainable AI
• Graph Neural Networks
• Interpretability Evaluation
• Graph Kernel Networks
• AIM Framework