When Synthetic Speech Is All You Have: Better Call GRPO

2026-07-09

AI Summary

This study addresses the challenge of adapting large language model–driven automatic speech recognition (ASR) systems in regulated domains such as banking, where access to real-world speech data is severely restricted due to privacy and compliance constraints. To overcome the acoustic distribution mismatch between synthetic and real speech, the authors propose the first application of the reward-free reinforcement learning algorithm GRPO (Group Relative Policy Optimization) for ASR adaptation using only synthetic data. By optimizing policies through rewards based on low word error rate (WER), GRPO reduces WER from 36.71% to 22.09%—a 40% relative improvement over supervised fine-tuning (SFT). Combining SFT with GRPO yields a further 45% reduction. The performance gain is attributed to behavioral policy optimization rather than changes in representation, thereby surpassing the limitations of conventional SFT.

Abstract

LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40\% relative to SFT (36.71\%$\to$22.09\%), and an SFT-then-GRPO combination pushes this further to 45\%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.

Problem

• synthetic speech
• automatic speech recognition
• acoustic mismatch
• privacy constraints
• domain adaptation

Innovation

• Group Relative Policy Optimization
• synthetic speech
• reinforcement learning
• automatic speech recognition
• supervised fine-tuning