Non-Autoregressive Minimum Bayes' Risk Decoding for Fast Speech Recognition

2026-06-16

AI Summary

This work addresses the performance limitations of non-autoregressive speech recognition, which stem from the absence of contextual modeling over previously generated tokens. The study introduces, for the first time, the Minimum Bayes Risk (MBR) criterion into this paradigm, proposing an efficient decoding framework based on parallel sampling and expected utility maximization. By leveraging the inherent parallelism of non-autoregressive models, the method generates multiple output hypotheses in a single forward pass to estimate risk and optimize the final prediction. Evaluated on LibriSpeech, Switchboard, AMI, and web-based lecture datasets, the approach consistently outperforms existing non-autoregressive methods while achieving faster decoding speeds than autoregressive counterparts.

Abstract

Non-autoregressive (NAR) decoding generates output tokens in parallel, making speech recognition faster than autoregressive decoding, which generates them sequentially from left to right. However, the recognition performance is degraded because NAR decoding cannot resolve uncertainty by conditioning on previously generated tokens. To address this issue, we propose a novel NAR decoding framework based on minimum Bayes' risk (MBR) decoding, termed NAR-MBR decoding, that maximizes the expected utility calculated from samples drawn from the output probability of an NAR model rather than maximizing the output probability. Notably, by leveraging the nature of NAR models, multiple samples are obtained efficiently with a single forward computation. Our experiments across LibriSpeech, Switchboard, AMI, and web presentation corpus demonstrated that our NAR-MBR decoding outperformed previous NAR decoding and ran faster than AR decoding.

Problem

• Non-autoregressive decoding
• speech recognition
• recognition performance
• uncertainty resolution

Innovation

• Non-Autoregressive Decoding
• Minimum Bayes' Risk
• Speech Recognition
• Parallel Generation
• Expected Utility