Beyond Prompting: Efficient and Robust Contextual Biasing for Speech LLMs via Logit-Space Integration (LOGIC)

2026-01-21

AI Summary

This work addresses the limitations of current speech large language models in recognizing dynamically added domain-specific entities—such as contacts or playlists—due to their reliance on static knowledge. Conventional prompting methods suffer from context-length constraints, high latency, and the “lost-in-the-middle” phenomenon, while post-hoc correction often induces hallucinations. To overcome these challenges, the authors propose LOGIC, a framework that directly injects contextual bias into the logit space during decoding, decoupling context integration from the input sequence. This approach achieves, for the first time, a constant-time biasing mechanism independent of input length, without modifying model architecture or increasing inference latency, and supports multilingual contextual biasing. Evaluated across 11 languages, LOGIC reduces entity word error rate (Entity WER) by 9% on average, with only a 0.30% increase in false alarm rate.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Statistical Relational/Logic Learning
• Knowledge Representation and Reasoning: Logic Programming

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The rapid emergence of new entities -- driven by cultural shifts, evolving trends, and personalized user data -- poses a significant challenge for existing Speech Large Language Models (Speech LLMs). While these models excel at general conversational tasks, their static training knowledge limits their ability to recognize domain-specific terms such as contact names, playlists, or technical jargon. Existing solutions primarily rely on prompting, which suffers from poor scalability: as the entity list grows, prompting encounters context window limitations, increased inference latency, and the"lost-in-the-middle"phenomenon. An alternative approach, Generative Error Correction (GEC), attempts to rewrite transcripts via post-processing but frequently suffers from"over-correction", introducing hallucinations of entities that were never spoken. In this work, we introduce LOGIC (Logit-Space Integration for Contextual Biasing), an efficient and robust framework that operates directly in the decoding layer. Unlike prompting, LOGIC decouples context injection from input processing, ensuring constant-time complexity relative to prompt length. Extensive experiments using the Phi-4-MM model across 11 multilingual locales demonstrate that LOGIC achieves an average 9% relative reduction in Entity WER with a negligible 0.30% increase in False Alarm Rate.

Problem

• Speech LLMs
• contextual biasing
• entity recognition
• prompting
• logit-space integration

Innovation

• Logit-Space Integration
• Contextual Biasing
• Speech LLMs
• Entity Recognition
• Efficient Decoding