Dynamic Lagging for Simultaneous Translation

2026-09-05

Abstract

In cascaded simultaneous speech translation, the machine translation (MT) system cannot control the read--write schedule of the upstream recognizer: it must decide, from a growing source prefix, how much target text to commit. We make a sentence-trained, decoder-only LLM prefix-aware by fine-tuning it on stable prefixes---the longest prefix that any translation up to the current partial source has shared with the model's own full-source output---mixed with full-sentence pairs, and prompt it through a single force-decode turn that carries the committed target forward as more source arrives, making the system flicker-free by construction. We fine-tune Qwen3-8B for EN to DE, JA, ZH, simulating the source stream with reference-transcript prefixes. Prefix finetuning preserves full-sentence quality while improving worst-position chunk quality, and it improves calibration of token-level commit confidence, reducing expected calibration error (ECE) on early source prefixes against a stable-prefix oracle. A single training-free threshold on that confidence is the most effective of the three latency controls we compare: it traces a continuous quality--latency frontier that outperforms the discrete wait-$k$ and target-suffix-deletion quality-latency tradeoff mechanisms. The effect holds well on FLEURS, WMT24++, and CoVoST~2 test sets, under both COMET and MetricX.

Problem

• Simultaneous Translation
• Machine Translation
• Latency Control
• Prefix Awareness

Innovation

• prefix-aware
• fine-tuning
• force-decode
• latency control
• calibration