A Joint-Distribution Route to Fair Representations with Continuous Sensitive Attributes

2026-08-11

AI Summary

This work addresses fair representation learning with continuous sensitive attributes by proposing a novel approach that avoids explicit estimation of conditional distributions. Instead, it directly measures the global discrepancy between the joint distribution of representations and sensitive attributes and the product of their marginal distributions to enforce statistical independence. Under decomposable witness function classes, this framework is equivalent to existing methods but offers superior computational efficiency. Specifically, the authors employ the Hilbert-Schmidt Independence Criterion (HSIC) to construct a closed-form statistic estimable in O(n²) time. The resulting FRHSIC algorithm achieves a fairness-accuracy trade-off comparable to baseline methods while significantly reducing per-iteration training time and attaining a faster convergence rate of O(n⁻¹/²).

Abstract

Fair representation learning with a continuous sensitive attribute $S$ requires a representation $Z$ that is statistically independent of $S$. Existing criteria, including generalized demographic parity, the expectation of integral probability metrics (EIPM), and mutual information, enforce this independence by averaging a per-value discrepancy between the conditional law $P_{Z \mid S=s}$ and the marginal $P_Z$ over the law of $S$. This approach requires a nonparametric surrogate for the conditional law at each sensitive value. We propose evaluating independence through a single joint discrepancy $d\left(P_{Z, S}, P_Z \otimes P_S\right)$ between the joint law and the product of its marginals. We establish a disintegration identity; on decomposable witness classes it equals the conditional-integral functional that EIPM and generalized demographic parity instantiate. By reaching the same target without the conditional law, this discrepancy can be estimated directly from samples via a dependence statistic rather than conditional smoothing. We take the Hilbert-Schmidt independence criterion (HSIC) as an instance of the joint discrepancy $d$ to investigate the statistical efficiency of replacing the conditional formulation. The HSIC estimator is a closed-form $O\left(n^2\right)$ statistic that converges at the $O\left(n^{-1 / 2}\right)$ rate, in contrast to the nonparametric $O\left(n^{-2 / 5}\right)$ rate of the conditional-route estimators. We prove this instance is equivalent to the conditional maximum mean discrepancy (MMD) integral up to an explicit spectral tail. The corresponding algorithmic implementation, i.e., FRHSIC, attains fairness-accuracy tradeoffs comparable to conditional-route basel es while reducing per-epoch training time.

Problem

• fair representation learning
• continuous sensitive attributes
• statistical independence
• joint distribution
• conditional law

Innovation

• joint distribution discrepancy
• continuous sensitive attributes
• Hilbert-Schmidt independence criterion
• fair representation learning
• statistical efficiency