Kolmogorov--Nagumo Mean Frameworks for Conditional Entropy

2026-05-08

AI Summary

This work addresses the limited expressiveness of existing conditional entropy frameworks—such as (η,F)-entropy—which fail to encompass important formulations like the Augustin–Csiszár entropy. To overcome this, the paper introduces, for the first time, the Kolmogorov–Nagumo (KN) averaging system into conditional entropy modeling, proposing the (η,ψ)-KN mean and establishing its equivalence to the η-mean. Building on this, a generalized g-conditional entropy framework is developed, substantially broadening the class of representable conditional entropies. This new framework not only subsumes the classical EAVG model but also precisely captures the Augustin–Csiszár conditional entropy. By integrating g-vulnerability theory, convex analysis, and key information-theoretic concepts—namely conditional random entropy (CRE) and the data processing inequality (DPI)—the study provides sufficient conditions under which the proposed framework satisfies CRE and DPI, thereby laying a more universal theoretical foundation for conditional entropy.

Abstract

We study conditional entropy frameworks based on the Kolmogorov--Nagumo (KN) mean. First, we introduce $(η, ψ)$-KN averaging (\texttt{EPKNAVG}), a KN-mean extension of the $η$-averaging (\texttt{EAVG}) framework for $(η, F)$-entropy, and prove that, under suitable concavification conditions, it is equivalent to \texttt{EAVG}. Second, motivated by generalized $g$-vulnerability, we propose a new framework of generalized $g$-conditional entropies. We show that this framework captures conditional entropies beyond the scope of \texttt{EAVG}-type representations. In particular, there exists $α\in(0,1)\cup(1,\infty)$ such that the Augustin--Csisz{\' a}r conditional entropy $H_α^{\mathrm{C}}(X|Y)$ cannot be represented by any $(η,F)$-entropy satisfying \texttt{EAVG}, whereas it is represented within the proposed framework. We further derive sufficient conditions for the proposed generalized $g$-conditional entropies to satisfy conditioning reduces entropy (\texttt{CRE}) and the data-processing inequality (\texttt{DPI}).

Problem

• conditional entropy
• Kolmogorov–Nagumo mean
• generalized g-vulnerability
• Augustin–Csiszár entropy
• entropy representation

Innovation

• Kolmogorov–Nagumo mean
• generalized g-conditional entropy
• Augustin–Csiszár entropy
• conditioning reduces entropy
• data-processing inequality