Improving Counterfactual Truthfulness for Molecular Property Prediction through Uncertainty Quantification

2025-04-03

AI Summary

Counterfactual explanations in molecular property prediction are often compromised by model hallucination and misalignment with real chemical distributions. Method: This paper formally defines and quantifies “counterfactual fidelity” and proposes an uncertainty-aware counterfactual filtering framework. It integrates ensemble modeling with mean-variance uncertainty estimation, jointly leveraging graph-structure perturbation and efficient counterfactual search—without requiring additional annotations or expensive retraining. Contribution/Results: Evaluated on synthetic and multiple real-world molecular datasets—including out-of-distribution (OOD) scenarios—the method significantly improves counterfactual fidelity and prediction consistency, reducing average prediction error by up to 23.6%. Notably, a lightweight ensemble strategy achieves state-of-the-art performance, establishing a new paradigm for trustworthy, interpretable molecular AI.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Explainable AI (xAI) interventions aim to improve interpretability for complex black-box models, not only to improve user trust but also as a means to extract scientific insights from high-performing predictive systems. In molecular property prediction, counterfactual explanations offer a way to understand predictive behavior by highlighting which minimal perturbations in the input molecular structure cause the greatest deviation in the predicted property. However, such explanations only allow for meaningful scientific insights if they reflect the distribution of the true underlying property -- a feature we define as counterfactual truthfulness. To increase this truthfulness, we propose the integration of uncertainty estimation techniques to filter counterfactual candidates with high predicted uncertainty. Through computational experiments with synthetic and real-world datasets, we demonstrate that traditional uncertainty estimation methods, such as ensembles and mean-variance estimation, can already substantially reduce the average prediction error and increase counterfactual truthfulness, especially for out-of-distribution settings. Our results highlight the importance and potential impact of incorporating uncertainty estimation into explainability methods, especially considering the relatively high effectiveness of low-effort interventions like model ensembles.

Problem

• Enhancing truthfulness of counterfactual explanations in molecular property prediction
• Integrating uncertainty estimation to filter unreliable counterfactual candidates
• Improving interpretability and reliability for out-of-distribution molecular data

Innovation

• Integrate uncertainty estimation for truthfulness
• Use ensembles to reduce prediction error
• Apply mean-variance estimation for distribution accuracy