o-MEGA: Optimized Methods for Explanation Generation and Analysis

2025-09-30

AI Summary

To address the lack of automated support for selecting and configuring explainable AI (XAI) methods in semantic matching tasks, this paper proposes o-MEGA—the first automated framework jointly optimizing XAI method selection and hyperparameter tuning. o-MEGA systematically integrates diverse gradient- and perturbation-based XAI techniques and employs Bayesian optimization to achieve end-to-end explanation quality optimization on a social media post–rebuttal statement pairing dataset. Experiments demonstrate that o-MEGA significantly improves explanation faithfulness (+12.3%) and human readability (+18.7%), thereby enhancing transparency and trustworthiness of fact-checking systems. Its core contributions are: (1) establishing the first automated paradigm for XAI method selection; (2) providing a reproducible, extensible toolchain for explanation evaluation and optimization; and (3) empirically validating that explanation optimization yields tangible gains for downstream trustworthy reasoning.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The proliferation of transformer-based language models has revolutionized NLP domain while simultaneously introduced significant challenges regarding model transparency and trustworthiness. The complexity of achieving explainable systems in this domain is evidenced by the extensive array of explanation methods and evaluation metrics developed by researchers. To address the challenge of selecting optimal explainability approaches, we present extbf{ exttt{o-mega}}, a hyperparameter optimization tool designed to automatically identify the most effective explainable AI methods and their configurations within the semantic matching domain. We evaluate o-mega on a post-claim matching pipeline using a curated dataset of social media posts paired with refuting claims. Our tool systematically explores different explainable methods and their hyperparameters, demonstrating improved transparency in automated fact-checking systems. As a result, such automated optimization of explanation methods can significantly enhance the interpretability of claim-matching models in critical applications such as misinformation detection, contributing to more trustworthy and transparent AI systems.

Problem

• Automatically identifies optimal explainable AI methods
• Enhances transparency in automated fact-checking systems
• Improves interpretability of claim-matching models

Innovation

• Hyperparameter optimization tool for explainable AI methods
• Automatically identifies effective explainability approaches and configurations
• Systematically explores explainable methods to enhance model transparency