Generalising maximum mean discrepancy: kernelised functional Bregman divergences

2026-04-27

AI Summary

This work addresses the limited integration of existing functional Bregman divergences with kernel methods and reproducing kernel Hilbert spaces (RKHS), which has hindered their applicability in modern machine learning. The paper presents the first systematic incorporation of the RKHS framework into functional Bregman divergences, leveraging the Riesz representation theorem and self-dual pairings to simplify their structure. Building upon kernel mean embeddings, the authors derive a computationally efficient form of the divergence. This approach not only establishes a theoretical bridge between Bregman geometry and kernel methods but also unifies existing techniques such as maximum mean discrepancy (MMD) within a common framework. Empirical evaluations demonstrate that the proposed divergence achieves strong performance in tasks including clustering, robust estimation, and generative modeling.

Technology Category

• Machine Learning: Kernel Methods
• Knowledge Representation and Reasoning: Other Foundations of Knowledge Representation & Reasoning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Bregman divergences play a pivotal role in statistics, machine learning and computational information geometry. Particularly in the context of machine learning, they are central to clustering, exponential families, parameter estimation and optimisation, among other things. Despite this, the full toolkit of Hilbert spaces and in particular reproducing kernel Hilbert spaces have not been systematically developed and applied to functional Bregman divergences, where points are functions rather than finite-dimensional parameter vectors. While other types of functional Bregman divergences have been studied, these are typically in a Banach space rather than more directly aligned with kernel methods and Hilbert-space geometry commonly used in machine learning. We consider functional Bregman divergences on a Hilbert space, where the self-dual pairing and Riesz representer afford us particularly convenient calculus. Further specialising Bregman generators as a composition involving a kernel mean embedding makes such divergences easy to estimate. We discuss applications in clustering, universal estimation, robust estimation and generative modelling, and contrast our approach with other types of Bregman divergences.

Problem

• Bregman divergences
• reproducing kernel Hilbert spaces
• functional data
• kernel methods
• Hilbert space geometry

Innovation

• functional Bregman divergence
• reproducing kernel Hilbert space
• kernel mean embedding
• Hilbert space geometry
• maximum mean discrepancy