MedSpeak: A Knowledge Graph-Aided ASR Error Correction Framework for Spoken Medical QA

2026-02-01

AI Summary

This work addresses the challenge of inaccurate automatic speech recognition (ASR) in medical question-answering scenarios, where errors in recognizing domain-specific terminology degrade downstream task performance. To this end, the authors propose MedSpeak, a novel framework that, for the first time, jointly integrates semantic information from a medical knowledge graph with the reasoning capabilities of large language models (LLMs) to correct ASR transcripts. By leveraging a multimodal fusion of acoustic, semantic, and phonetic features, MedSpeak significantly improves both medical term recognition accuracy and downstream question-answering performance across multiple healthcare benchmarks, achieving state-of-the-art results.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Spoken question-answering (SQA) systems relying on automatic speech recognition (ASR) often struggle with accurately recognizing medical terminology. To this end, we propose MedSpeak, a novel knowledge graph-aided ASR error correction framework that refines noisy transcripts and improves downstream answer prediction by leveraging both semantic relationships and phonetic information encoded in a medical knowledge graph, together with the reasoning power of LLMs. Comprehensive experimental results on benchmarks demonstrate that MedSpeak significantly improves the accuracy of medical term recognition and overall medical SQA performance, establishing MedSpeak as a state-of-the-art solution for medical SQA. The code is available at https://github.com/RainieLLM/MedSpeak.

Problem

• ASR error correction
• medical terminology
• spoken question-answering
• speech recognition
• medical QA

Innovation

• knowledge graph
• ASR error correction
• medical QA
• large language models
• spoken question answering