EffectiveASR: A Single-Step Non-Autoregressive Mandarin Speech Recognition Architecture with High Accuracy and Inference Speed

2024-06-13

AI Summary

To address the trade-off between accuracy and inference speed in non-autoregressive (NAR) speech recognition, this paper proposes EffectiveASR—a Chinese-oriented single-step NAR model. The core innovation is the Index Mapping Vector (IMV) alignment mechanism, which jointly models an alignment generator and an alignment predictor, and integrates end-to-end cross-entropy loss with alignment loss in a unified optimization framework. Unlike conventional iterative decoding or teacher-forced alignment strategies, EffectiveASR enables truly single-step, differentiable, and end-to-end training. On AISHELL-1, it achieves 4.26%/4.62% character error rate (CER), matching the accuracy of autoregressive Conformer while accelerating inference by approximately 30×. On AISHELL-2, it attains state-of-the-art performance, demonstrating both the effectiveness and generalizability of the IMV alignment mechanism for Chinese NAR speech recognition.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Non-autoregressive (NAR) automatic speech recognition (ASR) models predict tokens independently and simultaneously, bringing high inference speed. However, there is still a gap in the accuracy of the NAR models compared to the autoregressive (AR) models. In this paper, we propose a single-step NAR ASR architecture with high accuracy and inference speed, called EffectiveASR. It uses an Index Mapping Vector (IMV) based alignment generator to generate alignments during training, and an alignment predictor to learn the alignments for inference. It can be trained end-to-end (E2E) with cross-entropy loss combined with alignment loss. The proposed EffectiveASR achieves competitive results on the AISHELL-1 and AISHELL-2 Mandarin benchmarks compared to the leading models. Specifically, it achieves character error rates (CER) of 4.26%/4.62% on the AISHELL-1 dev/test dataset, which outperforms the AR Conformer with about 30x inference speedup.

Problem

• Non-autoregressive models
• Speech recognition
• Accuracy-speed trade-off

Innovation

• Non-autoregressive ASR
• Alignment Optimization
• End-to-end Training