🤖 AI Summary
Cross-attention maps in natural language inference (NLI) are widely assumed to be interpretable, yet their actual capacity to reveal sentence comparison and logical reasoning processes remains inadequately evaluated—particularly due to the high cost and limited scale of human-annotated explanations. Method: This work introduces, for the first time in NLI, a heuristic rule-based automatic explanation annotation method to overcome these bottlenecks, and conducts a comparative analysis between human and heuristic annotations on the eSNLI dataset. Contribution/Results: Experiments show strong positive correlation (ρ > 0.6) between heuristic and human annotations, validating the former’s utility for explanation quality evaluation. In contrast, raw RNN cross-attention weights exhibit only weak correlation (ρ ≈ 0.2) with human-validated explanations, exposing their limited interpretability. This study establishes a new benchmark, methodology, and empirical insight for assessing attention mechanism interpretability in NLI.
📝 Abstract
Attention maps in neural models for NLP are appealing to explain the decision made by a model, hopefully emphasizing words that justify the decision. While many empirical studies hint that attention maps can provide such justification from the analysis of sound examples, only a few assess the plausibility of explanations based on attention maps, i.e., the usefulness of attention maps for humans to understand the decision. These studies furthermore focus on text classification. In this paper, we report on a preliminary assessment of attention maps in a sentence comparison task, namely natural language inference. We compare the cross-attention weights between two RNN encoders with human-based and heuristic-based annotations on the eSNLI corpus. We show that the heuristic reasonably correlates with human annotations and can thus facilitate evaluation of plausible explanations in sentence comparison tasks. Raw attention weights however remain only loosely related to a plausible explanation.