Discrete Bridges for Mutual Information Estimation

2026-02-09

AI Summary

This work addresses the challenge of accurately estimating mutual information (MI) between discrete random variables by proposing a novel approach based on Discrete Bridge Matching. For the first time, discrete bridge models are introduced into MI estimation, reframing the problem as a domain translation task. The resulting DBMI estimator integrates generative modeling with information-theoretic techniques, specifically tailored for discrete data. Experimental results demonstrate that the proposed method significantly outperforms existing approaches in both low-dimensional and image-based MI estimation tasks, confirming its effectiveness and strong generalization capability.

Technology Category

• Machine Learning: Information Theory
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Diffusion bridge models in both continuous and discrete state spaces have recently become powerful tools in the field of generative modeling. In this work, we leverage the discrete state space formulation of bridge matching models to address another important problem in machine learning and information theory: the estimation of the mutual information (MI) between discrete random variables. By neatly framing MI estimation as a domain transfer problem, we construct a Discrete Bridge Mutual Information (DBMI) estimator suitable for discrete data, which poses difficulties for conventional MI estimators. We showcase the performance of our estimator on two MI estimation settings: low-dimensional and image-based.

Problem

• mutual information estimation
• discrete random variables
• discrete data
• information theory

Innovation

• Discrete Bridge
• Mutual Information Estimation
• Bridge Matching
• Domain Transfer
• Generative Modeling