What Was That Again? Certified Robustness for Automatic Speech Recognition

2026-06-25

AI Summary

This work addresses the vulnerability of current automatic speech recognition (ASR) systems to both adversarial and benign perturbations, as well as the difficulty of anomaly detection in deployment scenarios lacking ground-truth transcriptions. To this end, the authors propose a dual-gate diagnostic framework grounded in a certifiable dual-atomic auditing mechanism. The approach leverages statistical wealth accumulation to verify token presence and adversarial exclusion, and employs a ranking tournament to select the optimal output sequence, thereby enabling fine-grained certification at both word and sentence levels. Notably, the method significantly enhances acoustic robustness without requiring reference transcripts, achieving up to a 55% relative reduction in word error rate across four mainstream ASR architectures while simultaneously improving recall and weakening the Spearman correlation between confidence scores and word error rates.

Abstract

Automatic Speech Recognition systems are notoriously both sensitive to adversarial and benign perturbations. While this has been repeatedly demonstrated using reference datasets, detecting such behaviors in deployed systems is incredibly challenging, due to the absence of oracle knowledge of the true transcription. We demonstrate that employing a certification-inspired mechanism can significantly decrease WER, increase recall, and decrease the Spearman correlation between confidence and WER. We achieve this through a dual-gate diagnostic pipeline: a Two-Sided Atomic Audit that accumulates statistical wealth to certify both token existence and adversarial exclusion, and a Rank-Based Tournament that selects the winning sequence. Our evaluations across four diverse architectures demonstrate up to a 55% relative reduction in Word Error Rate, while also providing granular word- and sentence-level certifications to enhance acoustic security.

Problem

• Automatic Speech Recognition
• adversarial perturbations
• robustness
• Word Error Rate
• acoustic security

Innovation

• certified robustness
• automatic speech recognition
• adversarial perturbations
• dual-gate diagnostic pipeline
• word error rate