Reweighted information inequalities

2026-03-13

AI Summary

This work addresses the challenge of characterizing the relationship between Fisher information and relative entropy or Wasserstein distance in non-log-concave, multimodal distributions. The authors propose a reweighting framework that extends log-Sobolev inequalities and transportation-information inequalities to mixture distributions for the first time. They establish that if a distribution is close to a mixture in terms of relative Fisher information, then it is also close—measured by relative entropy or Wasserstein distance—to a reweighted version of one of the mixture components. This result not only provides interpretable Fisher information bounds for non-log-concave measures but also offers theoretical insight into the behavior of Langevin Monte Carlo methods in multimodal sampling scenarios.

Technology Category

• Machine Learning: Information Theory
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements

Abstract

We establish a variant of the log-Sobolev and transport-information inequalities for mixture distributions. If a probability measure $π$ can be decomposed into components that individually satisfy such inequalities, then any measure $μ$ close to $π$ in relative Fisher information is close in relative entropy or transport distance to a reweighted version of $π$ with the same mixture components but possibly different weights. This provides a user-friendly interpretation of Fisher information bounds for non-log-concave measures and explains phenomena observed in the analysis of Langevin Monte Carlo for multimodal distributions.

Problem

• log-Sobolev inequality
• transport-information inequality
• mixture distributions
• Fisher information
• Langevin Monte Carlo

Innovation

• reweighted information inequalities
• log-Sobolev inequality
• transport-information inequality
• mixture distributions
• relative Fisher information