Agreement-Constrained Probabilistic Minimum Bayes Risk Decoding

2025-12-01

AI Summary

To address the quadratic time complexity in Minimum Bayes Risk (MBR) decoding arising from pairwise utility evaluation of all candidate hypotheses, this paper proposes Probabilistic MBR (PMBR) decoding. The core innovation lies in leveraging a knowledge distillation model to guide probabilistic completion of the utility matrix, coupled with consistency constraints to mitigate approximation error. By avoiding explicit computation of utilities for all candidate pairs, PMBR significantly reduces computational overhead while preserving decoding quality. On the WMT’23 English–German bidirectional translation tasks, PMBR achieves up to a threefold reduction in utility matrix completion error at comparable computational cost, and consistently outperforms baseline methods in both BLEU and COMET scores. This demonstrates a superior efficiency–quality trade-off over conventional MBR decoding.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Minimum Bayes risk (MBR) decoding generates high-quality translations by maximizing the expected utility of output candidates, but it evaluates all pairwise scores over the candidate set; hence, it takes quadratic time with respect to the number of candidates. To reduce the number of utility function calls, probabilistic MBR (PMBR) decoding partially evaluates quality scores using sampled pairs of candidates and completes the missing scores with a matrix completion algorithm. Nevertheless, it degrades the translation quality as the number of utility function calls is reduced. Therefore, to improve the trade-off between quality and cost, we propose agreement-constrained PMBR (AC-PMBR) decoding, which leverages a knowledge distilled model to guide the completion of the score matrix. Our AC-PMBR decoding improved approximation errors of matrix completion by up to 3 times and achieved higher translation quality compared with PMBR decoding at a comparable computational cost on the WMT'23 En$leftrightarrow$De translation tasks.

Problem

• Improves translation quality in probabilistic MBR decoding
• Reduces computational cost by optimizing matrix completion
• Enhances trade-off between quality and efficiency in machine translation

Innovation

• Agreement-constrained PMBR decoding improves matrix completion
• Uses knowledge-distilled model to guide score matrix completion
• Reduces approximation errors and enhances translation quality efficiently