COALA: Robust Contextualized Speech-augmented Language Modeling for ASR via Contrastive Regularizer and Biasing Score Estimation

2026-07-09

AI Summary

This work addresses the challenges of limited context windows and training instability in automatic speech recognition under multi-domain entity co-occurrence scenarios. To this end, the authors propose the COALA framework, which maps the hidden representations of a speech-augmented language model into a discriminative embedding space to quantify the alignment strength between audio segments and candidate entities. By integrating contrastive learning regularization with a bias scoring estimation mechanism, COALA significantly enhances the robustness and stability of contextual biasing. The method effectively mitigates collapse issues commonly observed in multi-objective training and achieves state-of-the-art contextual biasing performance across varying bias lexicon sizes on the LibriSpeech benchmark, demonstrating its efficacy and strong generalization capability.

Abstract

Contextual biasing seeks to integrate external knowledge into automatic speech recognition (ASR) systems to accurately recognize domain-specific entities. In this paper, we propose COALA (Contextualized ASR Leveraging Biasing Scoring), a robust framework designed to enhance speech-augmented language models (SLMs) in complex multi-entity scenarios. Considering the inherent context-window limitations of SLMs, identifying relevant target entities from a large-scale biasing list is crucial for effective recognition. To this end, COALA maps SLM latent representations into a specialized discriminative space to quantify the matching intensity between audio segments and candidate entities. Furthermore, we address the training collapse in prior study when handling multi-target utterances-where multiple rare words co-occur. Experimental results on the LibriSpeech benchmark demonstrate that COALA consistently achieves superior contextual biasing performance across various biasing list scales.

Problem

• contextual biasing
• automatic speech recognition
• domain-specific entities
• multi-entity scenarios
• biasing list

Innovation

• contextual biasing
• speech-augmented language model
• contrastive regularizer
• biasing score estimation
• multi-entity ASR