B-XAIC Dataset: Benchmarking Explainable AI for Graph Neural Networks Using Chemical Data

2025-05-28

AI Summary

Existing GNN interpretability evaluation frameworks heavily rely on synthetic data and proxy metrics, failing to reflect the faithfulness of explanations in real-world chemical scenarios. Method: We propose B-XAIC—the first XAI benchmark for molecular graphs—built on real molecular datasets and tasks, incorporating expert-validated ground-truth rationale annotations. Our methodology integrates molecular graph representation learning, systematic evaluation of GNN explanation algorithms, domain-expert annotation, and a rigorous consistency verification protocol. Contribution/Results: Experiments reveal widespread faithfulness deficits among mainstream GNN explanation methods on chemical tasks. B-XAIC establishes the first rationale-driven, chemistry-specific interpretability evaluation paradigm, providing a standardized, reproducible testing platform for XAI algorithms. It significantly enhances model trustworthiness and debuggability in molecular machine learning, enabling rigorous, domain-grounded assessment of explanation quality.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Understanding the reasoning behind deep learning model predictions is crucial in cheminformatics and drug discovery, where molecular design determines their properties. However, current evaluation frameworks for Explainable AI (XAI) in this domain often rely on artificial datasets or simplified tasks, employing data-derived metrics that fail to capture the complexity of real-world scenarios and lack a direct link to explanation faithfulness. To address this, we introduce B-XAIC, a novel benchmark constructed from real-world molecular data and diverse tasks with known ground-truth rationales for assigned labels. Through a comprehensive evaluation using B-XAIC, we reveal limitations of existing XAI methods for Graph Neural Networks (GNNs) in the molecular domain. This benchmark provides a valuable resource for gaining deeper insights into the faithfulness of XAI, facilitating the development of more reliable and interpretable models.

Problem

• Evaluating XAI for GNNs in molecular domains lacks real-world complexity
• Current XAI benchmarks use artificial data, missing explanation faithfulness
• B-XAIC provides real molecular data with ground-truth rationales for evaluation

Innovation

• Real-world molecular data benchmark
• Diverse tasks with ground-truth rationales
• Comprehensive XAI evaluation for GNNs