Group Entropies and Mirror Duality: A Class of Flexible Mirror Descent Updates for Machine Learning

2026-03-09

AI Summary

This work addresses the limitations of traditional mirror descent algorithms in adapting to diverse geometric structures and statistical distributions inherent in data. The authors propose a unified framework grounded in formal group theory and group entropy, which employs group-theoretic logarithm and exponential functions to define flexible mirror maps. A key innovation is the introduction of a “mirror duality” mechanism that enables dynamic switching between a link function and its inverse. This framework yields an infinitely tunable family of mirror descent algorithms, substantially expanding the design space for regularizers and natural gradients. Experimental results on large-scale simplex-constrained quadratic programming problems demonstrate that the proposed method consistently outperforms existing approaches in terms of convergence speed, robustness, and adaptability.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization
• Game Theory and Economic Paradigms: Mechanism Design

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

We introduce a comprehensive theoretical and algorithmic framework that bridges formal group theory and group entropies with modern machine learning, paving the way for an infinite, flexible family of Mirror Descent (MD) optimization algorithms. Our approach exploits the rich structure of group entropies, which are generalized entropic functionals governed by group composition laws, encompassing and significantly extending all trace-form entropies such as the Shannon, Tsallis, and Kaniadakis families. By leveraging group-theoretical mirror maps (or link functions) in MD, expressed via multi-parametric generalized logarithms and their inverses (group exponentials), we achieve highly flexible and adaptable MD updates that can be tailored to diverse data geometries and statistical distributions. To this end, we introduce the notion of \textit{mirror duality}, which allows us to seamlessly switch or interchange group-theoretical link functions with their inverses, subject to specific learning rate constraints. By tuning or learning the hyperparameters of the group logarithms enables us to adapt the model to the statistical properties of the training distribution, while simultaneously ensuring desirable convergence characteristics via fine-tuning. This generality not only provides greater flexibility and improved convergence properties, but also opens new perspectives for applications in machine learning and deep learning by expanding the design of regularizers and natural gradient algorithms. We extensively evaluate the validity, robustness, and performance of the proposed updates on large-scale, simplex-constrained quadratic programming problems.

Problem

• Mirror Descent
• Group Entropies
• Optimization Algorithms
• Machine Learning
• Statistical Distributions

Innovation

• Group Entropies
• Mirror Descent
• Mirror Duality
• Generalized Logarithms
• Formal Group Theory