Explainable AI in Speaker Recognition -- Attention Map Visualisation and Evaluation

2026-06-22

AI Summary

This study addresses the lack of systematic and empirically validated evaluation methods for attention maps in speaker recognition tasks. To this end, we propose a modified Random Input Sampling for Evaluation (Modified RISE-eval) algorithm that enables more accurate quantification of attention map quality and facilitates a systematic comparison between GradCAM and LayerCAM under varying conditions. Experimental results demonstrate that both methods exhibit distinct strengths, and the proposed evaluation framework effectively uncovers their underlying decision-making rationales. The findings substantiate the practical utility and reliability of our approach in assessing attention mechanisms within speaker recognition systems.

Abstract

Explaining and understanding the decision-making process of artificial intelligence (AI) systems, particularly those implemented by neural networks, falls within the field of explainable AI (XAI). Analogous to the human attention mechanism, neural networks are assumed to possess their own attention mechanisms that selectively process information during decision-making. This work proposes to study one XAI topic: analysing and visualising the attention mechanisms of neural networks. Our experiments are performed on speaker recognition neural networks that are trained to identify speaker identity from a given utterance. Previous studies have widely used class activation map (CAM)-based methods to analyse and visualise the attention mechanisms of neural networks. Each of these methods produces an attention map for each network input, highlighting which input regions are selectively processed when the speaker recognition network makes decisions. However, the evaluation of attention maps produced by these methods remains largely underexplored. This work systematically reviews an existing attention map evaluation algorithm, establishing key concepts and identifying its shortcomings. On the basis of this existing evaluation algorithm, a new version is then proposed to address the identified shortcomings, called the Modified Randomised Input Sampling for Explanation - Evaluation algorithm (Modified RISE-eval). Using Modified RISE-eval, we evaluate the attention maps produced by two representative CAM-based methods, GradCAM and LayerCAM, applied to a certain speaker recognition network. The evaluation results demonstrate that GradCAM and LayerCAM each exhibit distinct advantages when applied under different experimental conditions in the speaker recognition task.

Problem

• Explainable AI
• Speaker Recognition
• Attention Map
• Evaluation
• Class Activation Map

Innovation

• Explainable AI
• Attention Map Evaluation
• Modified RISE-eval
• Speaker Recognition
• Class Activation Mapping