Faster Machine Translation Ensembling with Reinforcement Learning and Competitive Correction

2025-01-25

AI Summary

To address the high computational overhead and performance bottlenecks arising from the “select-then-fuse” paradigm in neural machine translation (NMT) ensemble decoding, this paper proposes SmartGen—a reinforcement learning–based end-to-end intelligent ensemble framework. Methodologically, SmartGen integrates candidate selection and fusion into a single trainable process via a Deep Q-Network (DQN), enabling low-latency, differentiable decision-making. It further introduces a Competitive Correction Block (CCB) to dynamically refine translations and unifies encoder-decoder fusion with cross-model collaboration within a coherent architecture. Empirical evaluation on English–Indian language pair translation demonstrates that SmartGen achieves BLEU scores comparable to full ensembles of eight models (L=8) using only 2–3 candidate models. Moreover, it delivers a 3.2× speedup in inference latency and reduces GPU memory consumption by 67%, striking a superior balance between efficiency and translation quality.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Ensemble Methods
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Ensembling neural machine translation (NMT) models to produce higher-quality translations than the $L$ individual models has been extensively studied. Recent methods typically employ a candidate selection block (CSB) and an encoder-decoder fusion block (FB), requiring inference across extit{all} candidate models, leading to significant computational overhead, generally $Omega(L)$. This paper introduces extbf{SmartGen}, a reinforcement learning (RL)-based strategy that improves the CSB by selecting a small, fixed number of candidates and identifying optimal groups to pass to the fusion block for each input sentence. Furthermore, previously, the CSB and FB were trained independently, leading to suboptimal NMT performance. Our DQN-based extbf{SmartGen} addresses this by using feedback from the FB block as a reward during training. We also resolve a key issue in earlier methods, where candidates were passed to the FB without modification, by introducing a Competitive Correction Block (CCB). Finally, we validate our approach with extensive experiments on English-Hindi translation tasks in both directions.

Problem

• Machine Translation
• Resource Consumption
• Translation Quality Optimization

Innovation

• SmartGen
• Reinforcement Learning
• Translation Quality Enhancement