Interpreting Multi-Branch Anti-Spoofing Architectures: Correlating Internal Strategy with Empirical Performance

2026-01-22

Mathematics

AI Summary

This work addresses the opacity of internal decision mechanisms in multi-branch audio anti-spoofing models and the lack of effective characterization of inter-branch collaboration. For the first time, the internal strategies of the AASIST3 model are formalized into four operational prototypes. Spectral signatures are constructed via the dominant eigenvalues of covariance operators, and a CatBoost meta-classifier combined with TreeSHAP attribution is employed to quantify each branch’s contribution and confidence (Cb). Experiments on 13 attack types from ASVspoof 2019 reveal phenomena of “effective specialization” (e.g., A09 with EER of 0.04%) and “defective specialization” (e.g., A17/A18 with EERs of 14.26%/28.63%), establishing a direct link between internal structural behavior and empirical performance while exposing structural dependencies that conventional evaluation metrics fail to capture.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Multi-branch deep neural networks like AASIST3 achieve state-of-the-art comparable performance in audio anti-spoofing, yet their internal decision dynamics remain opaque compared to traditional input-level saliency methods. While existing interpretability efforts largely focus on visualizing input artifacts, the way individual architectural branches cooperate or compete under different spoofing attacks is not well characterized. This paper develops a framework for interpreting AASIST3 at the component level. Intermediate activations from fourteen branches and global attention modules are modeled with covariance operators whose leading eigenvalues form low-dimensional spectral signatures. These signatures train a CatBoost meta-classifier to generate TreeSHAP-based branch attributions, which we convert into normalized contribution shares and confidence scores (Cb) to quantify the model’s operational strategy. By analyzing 13 spoofing attacks from the ASVspoof 2019 benchmark, we identify four operational archetypes—ranging from “Effective Specialization” (e.g., A09, Equal Error Rate (EER) 0.04%, C=1.56) to “Ineffective Consensus” (e.g., A08, EER 3.14%, C=0.33). Crucially, our analysis exposes a “Flawed Specialization” mode where the model places high confidence in an incorrect branch, leading to severe performance degradation for attacks A17 and A18 (EER 14.26% and 28.63%, respectively). These quantitative findings link internal architectural strategy directly to empirical reliability, highlighting specific structural dependencies that standard performance metrics overlook.

Problem

• multi-branch architectures
• audio anti-spoofing
• interpretability
• spoofing attacks
• decision dynamics

Innovation

• multi-branch interpretability
• spectral signatures
• TreeSHAP attribution
• anti-spoofing architecture
• component-level analysis