Correlation of divergency: c-delta. Being different in a similar way or not

2025-10-19

AI Summary

This paper addresses the problem of quantifying similarity in internal dispersion patterns between two sets of numerical data. We propose a novel metric—the *c-delta coefficient*—which operates outside conventional correlation frameworks and is the first to formalize structural association defined as “differing in similar ways.” The method computes, for each observation, its mean absolute deviation from all other values within the same group, then applies root-mean-square normalization to ensure scale invariance. The c-delta coefficient offers strong interpretability and robustness across scales, enabling principled comparison of variability structures across heterogeneous systems (e.g., human vs. model responses). We validate its efficacy across quantum physics, genomics, and machine learning applications. The metric provides a theoretically grounded, domain-agnostic tool for comparing intrinsic variability in complex, multidisciplinary datasets.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Analogy
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This paper introduces the correlation-of-divergency coefficient, c-delta, a custom statistical measure designed to quantify the similarity of internal divergence patterns between two groups of values. Unlike conventional correlation coefficients such as Pearson or Spearman, which assess the association between paired values, c-delta evaluates whether the way values differ within one group is mirrored in another. The method involves calculating, for each value, its divergence from all other values in its group, and then comparing these patterns across the two groups (e.g., human vs machine intelligence). The coefficient is normalised by the average root mean square divergence within each group, ensuring scale invariance. Potential applications of c-delta span quantum physics, where it can compare the spread of measurement outcomes between quantum systems, as well as fields such as genetics, ecology, psychometrics, manufacturing, machine learning, and social network analysis. The measure is particularly useful for benchmarking, clustering validation, and assessing the similarity of variability structures. While c-delta is not bounded between -1 and 1 and may be sensitive to outliers (but so is PMCC), it offers a new perspective for analysing internal variability and divergence. The article discusses the mathematical formulation, potential adaptations for complex data, and the interpretative considerations relevant to this alternative approach.

Problem

• Quantifying similarity of internal divergence patterns between groups
• Comparing variability structures across different measurement systems
• Assessing mirrored divergence in fields like quantum physics

Innovation

• Introduces correlation-of-divergency coefficient c-delta
• Compares internal divergence patterns between value groups
• Normalized measure ensures scale invariance across applications