A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization

2026-02-02

AI Summary

This work addresses the non-convergence, numerical instability, and slow convergence issues arising from the Log-E-Exp objective in compositional entropy risk minimization by proposing SCENT, a geometry-aware stochastic algorithm. SCENT reformulates the original problem into a min-min dual form and introduces, for the first time, a stochastic proximal mirror descent (SPMD) update for the dual variables based on a Bregman divergence induced by the negative exponential function. Theoretical analysis shows that under convex settings, SCENT achieves a convergence rate of $O(1/\sqrt{T})$, outperforming standard stochastic gradient descent. Empirical evaluations demonstrate that SCENT significantly surpasses existing baselines across diverse tasks—including extreme classification, partial AUC maximization, contrastive learning, and distributionally robust optimization—delivering stable and efficient convergence.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

This paper studies optimization for a family of problems termed $\textbf{compositional entropic risk minimization}$, in which each data's loss is formulated as a Log-Expectation-Exponential (Log-E-Exp) function. The Log-E-Exp formulation serves as an abstraction of the Log-Sum-Exponential (LogSumExp) function when the explicit summation inside the logarithm is taken over a gigantic number of items and is therefore expensive to evaluate. While entropic risk objectives of this form arise in many machine learning problems, existing optimization algorithms suffer from several fundamental limitations including non-convergence, numerical instability, and slow convergence rates. To address these limitations, we propose a geometry-aware stochastic algorithm, termed $\textbf{SCENT}$, for the dual formulation of entropic risk minimization cast as a min--min optimization problem. The key to our design is a $\textbf{stochastic proximal mirror descent (SPMD)}$ update for the dual variable, equipped with a Bregman divergence induced by a negative exponential function that faithfully captures the geometry of the objective. Our main contributions are threefold: (i) we establish an $O(1/\sqrt{T})$ convergence rate of the proposed SCENT algorithm for convex problems; (ii) we theoretically characterize the advantages of SPMD over standard SGD update for optimizing the dual variable; and (iii) we demonstrate the empirical effectiveness of SCENT on extreme classification, partial AUC maximization, contrastive learning and distributionally robust optimization, where it consistently outperforms existing baselines.

Problem

• compositional entropic risk minimization
• Log-Expectation-Exponential
• optimization
• numerical instability
• convergence

Innovation

• compositional entropic risk minimization
• stochastic proximal mirror descent
• geometry-aware optimization
• Bregman divergence
• Log-Expectation-Exponential