EMPEROR: Efficient Moment-Preserving Representation of Distributions

2025-09-19

AI Summary

To address the low representational efficiency of high-dimensional probability distributions in neural networks and the loss of statistical details inherent in conventional global pooling, this paper proposes EMPEROR: a framework that projects high-dimensional features onto random directions based on sliced moment theory, fits lightweight one-dimensional Gaussian mixture models (GMMs), and encodes their statistical moments into compact distribution descriptors. For the first time, it rigorously unifies the Carleman condition with the Cramér–Wold theorem to guarantee distributional uniqueness and derives an optimal finite-sample error bound. Empirically, EMPEROR significantly outperforms heuristic pooling methods—including mean and max pooling—across multimodal tasks (e.g., image classification, point cloud analysis, and time-series modeling), achieving superior expressiveness, computational efficiency, and theoretical soundness.

Technology Category

• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Representation Learning for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce EMPEROR (Efficient Moment-Preserving Representation of Distributions), a mathematically rigorous and computationally efficient framework for representing high-dimensional probability measures arising in neural network representations. Unlike heuristic global pooling operations, EMPEROR encodes a feature distribution through its statistical moments. Our approach leverages the theory of sliced moments: features are projected onto multiple directions, lightweight univariate Gaussian mixture models (GMMs) are fit to each projection, and the resulting slice parameters are aggregated into a compact descriptor. We establish determinacy guarantees via Carleman's condition and the Cramér-Wold theorem, ensuring that the GMM is uniquely determined by its sliced moments, and we derive finite-sample error bounds that scale optimally with the number of slices and samples. Empirically, EMPEROR captures richer distributional information than common pooling schemes across various data modalities, while remaining computationally efficient and broadly applicable.

Problem

• Representing high-dimensional probability measures in neural networks
• Encoding feature distributions through statistical moments efficiently
• Capturing richer distributional information than heuristic pooling operations

Innovation

• Encodes distributions through statistical moments
• Uses sliced projections and Gaussian mixture models
• Provides determinacy guarantees and error bounds