Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study

2025-08-13

AI Summary

Manual annotation of textual explanations for interpretable NLP is costly and inherently unscalable. Method: We propose a multi-LLM collaborative framework for automated explanation generation to enhance natural language inference (NLI) classifiers. It integrates outputs from multiple state-of-the-art large language models to produce high-quality, faithful reasoning rationales; employs NLG evaluation metrics to assess explanation quality; and fine-tunes downstream NLI classifiers—specifically on the SNLI and MNLI benchmarks—using these generated explanations as auxiliary supervision. Contribution/Results: Explanations automatically generated by LLMs significantly improve pre-trained NLI model performance, matching the efficacy of human-annotated explanations. This work provides the first empirical validation of the effectiveness and scalability of *automatically generated* explanations for model enhancement. By eliminating reliance on manual annotation, it establishes a novel, scalable paradigm for interpretable NLP.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

In the rapidly evolving field of Explainable Natural Language Processing (NLP), textual explanations, i.e., human-like rationales, are pivotal for explaining model predictions and enriching datasets with interpretable labels. Traditional approaches rely on human annotation, which is costly, labor-intensive, and impedes scalability. In this work, we present an automated framework that leverages multiple state-of-the-art large language models (LLMs) to generate high-quality textual explanations. We rigorously assess the quality of these LLM-generated explanations using a comprehensive suite of Natural Language Generation (NLG) metrics. Furthermore, we investigate the downstream impact of these explanations on the performance of pre-trained language models (PLMs) and LLMs across natural language inference tasks on two diverse benchmark datasets. Our experiments demonstrate that automated explanations exhibit highly competitive effectiveness compared to human-annotated explanations in improving model performance. Our findings underscore a promising avenue for scalable, automated LLM-based textual explanation generation for extending NLP datasets and enhancing model performance.

Problem

• Evaluating if LLM-generated explanations improve classification performance
• Automating textual explanations to replace costly human annotations
• Assessing impact of automated explanations on model task performance

Innovation

• Leveraging multiple LLMs for explanation generation
• Automated framework replaces costly human annotation
• Comprehensive NLG metrics assess explanation quality