Conversational Domain Adaptation of IndicTrans2 across 21 Indic Languages via Experience Replay and Model Soups

2026-06-27

AI Summary

This study addresses the challenge that IndicTrans2, trained on general-domain text, struggles to effectively translate colloquial dialogue, while direct fine-tuning compromises its general-purpose translation performance. The authors present the first systematic evaluation of experience replay and Model Soups for domain adaptation to conversational text across 21 Indian languages, adapting IndicTrans2-1B using only publicly available dialogue corpora (OpenSubtitles, BPCC-H-Daily, and Tatoeba). Their approach yields an average improvement of +6.2 chrF on conversational test sets across all languages, while maintaining near-equivalent performance on the FLORES general-domain benchmark (mean change: −0.17 chrF, with fluctuations under 0.7 chrF). The gains are statistically significant (p ≤ 0.004) and primarily attributed to better register alignment rather than improvements in subjective translation quality.

Abstract

IndicTrans2 is the strongest open English to Indic translation system, but like most systems it is trained on general text and tends to sound stiff on casual, conversational input. We adapt IndicTrans2-1B to conversational register across all 21 Indic languages using only public data (OpenSubtitles, BPCC-H-Daily, Tatoeba). Plain fine-tuning improves conversational chrF but forgets the general domain (it drops 3.9 chrF on FLORES for Hindi). Mixing general data back into training (experience replay) and then averaging the fine-tuned weights with the base (model souping) removes that trade-off: the resulting model beats IndicTrans2-1B on conversational chrF in every one of the 21 languages (mean +6.2) while matching it on FLORES (mean change -0.17, all within 0.7 chrF). Paired bootstrap tests confirm the conversational gains are significant (p <= 0.004) and that FLORES is not significantly degraded. We are deliberate about scope: these are chrF gains, and a blind human plus multi-model LLM check does not confirm them as a perceived quality improvement, so we treat the conversational gain as largely a register match to the references rather than proof of better translation. The techniques are not new; the contribution is the honest, end-to-end study in the Indic conversational setting.

Problem

• conversational domain adaptation
• Indic languages
• machine translation
• register adaptation
• domain shift

Innovation

• experience replay
• model soups
• conversational machine translation
• domain adaptation
• Indic languages