Dual Representation of Minimum Divergence Under Integral Constraints

2026-03-21

AI Summary

This work investigates the dual representation of the minimum f-divergence problem under integral constraints, a formulation with broad applications in sequential inference, multi-armed bandits, and distributionally robust optimization. Focusing on the observation space $[0,1]^K$, the authors propose a two-stage approach: first deriving a finite-dimensional convex dual for distributions with finite support, then extending the result to arbitrary distributions via an abstract interchange argument. This framework generalizes the well-known one-dimensional KL_inf duality under mean constraints to higher dimensions and to broader classes of f-divergences and integral constraint functions. The resulting theory substantially expands the scope of existing results and enables the construction of near-optimal statistical procedures—approaching theoretical lower bounds—for sequential testing, estimation, and change-point detection.

Technology Category

• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Minimum divergence problems under integral constraints appear throughout statistics and probability, including sequential inference, bandit theory, and distributionally robust optimization. In many such settings, dual representations are the key step that convert information-theoretic lower bounds into computationally tractable (and often near-optimal) algorithms. In this paper, we present a general two-stage recipe for deriving dual representations of constrained minimum divergence (in the second argument) for distributions supported on $[0,1]^K$. The first stage derives a dual representation for finitely-supported distributions using classical finite-dimensional convex duality techniques, while the second establishes an abstract interchange argument that lifts this discretized dual to arbitrary distributions. We begin with the simplest case of mean-constrained minimum relative entropy, commonly called $\mathrm{KL}_{\inf}$, and generalize an existing argument from multi-armed bandits literature for $K=1$ to arbitrary dimensions. Our main contribution is to significantly expand the scope of this approach to a broad class of $f$-divergences (beyond relative entropy) and to general integral constraint functionals (beyond the mean constraint). Finally, we illustrate the statistical implications of our results by constructing optimal procedures for sequential testing, estimation, and change detection with observations in $[0,1]^K$.

Problem

• minimum divergence
• integral constraints
• dual representation
• f-divergences
• distributionally robust optimization

Innovation

• dual representation
• f-divergence
• integral constraints
• convex duality
• sequential inference