Shift-Dispersion Decompositions of Wasserstein and Cram'er Distances

2024-08-19

AI Summary

Wasserstein and Cramér distances lack directional interpretability—i.e., they do not distinguish between location shifts and scale changes in distributional discrepancies. To address this, we propose an interpretable framework based on geometric decomposition of quantile functions, uniquely disentangling each distance into directed shift (location) and dispersion (scale) components. This decomposition is the first to satisfy naturalness, uniqueness, and additivity—key statistical desiderata—within the location-scale family. We further derive explicit sensitivity expressions of the distances with respect to location and dispersion parameters, and establish a weak stochastic order theory that jointly characterizes both location and dispersion orderings. Empirically validated on extreme temperature forecasting evaluation and probabilistic survey design in economics, our method substantially enhances semantic clarity in interpreting distributional differences and strengthens decision-support capabilities.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Divergence functions are measures of distance or dissimilarity between probability distributions that serve various purposes in statistics and applications. We propose decompositions of Wasserstein and Cram'er distances$-$which compare two distributions by integrating over their differences in distribution or quantile functions$-$into directed shift and dispersion components. These components are obtained by dividing the differences between the quantile functions into contributions arising from shift and dispersion, respectively. Our decompositions add information on how the distributions differ in a condensed form and consequently enhance the interpretability of the underlying divergences. We show that our decompositions satisfy a number of natural properties and are unique in doing so in location-scale families. The decompositions allow to derive sensitivities of the divergence measures to changes in location and dispersion, and they give rise to weak stochastic order relations that are linked to the usual stochastic and the dispersive order. Our theoretical developments are illustrated in two applications, where we focus on forecast evaluation of temperature extremes and on the design of probabilistic surveys in economics.

Problem

• Decompose Wasserstein and Cramér distances into shift and dispersion components
• Enhance interpretability of divergence measures between probability distributions
• Apply decompositions to forecast evaluation and probabilistic survey design

Innovation

• Decompose Wasserstein and Cramér distances into shift and dispersion
• Enhance interpretability of divergence measures via decomposition
• Apply decomposition to forecast evaluation and survey design