Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations

2024-06-22

arXiv.org

AI Summary

This work addresses the challenge of quantifying nonlinear inter-manifold correlations in high-dimensional multimodal data. We propose a geometry-driven, intrinsic-dimension-based metric that treats intrinsic dimensionality as a proxy signal for nonlinear dependence—marking the first such use of intrinsic dimension for this purpose. Unlike conventional linear (e.g., CCA), low-dimensional, or kernel-based methods (e.g., HSIC), our approach enables unsupervised, interpretable quantification of strong nonlinear associations between high-dimensional manifolds, such as vision–text embeddings. The method integrates manifold learning techniques (MIND, TwoNN) with nonparametric correlation analysis and demonstrates robustness on synthetic benchmarks. Empirical evaluation on CLIP and FLAVA reveals previously undetected strong cross-modal manifold correlations, significantly outperforming Pearson correlation, CCA, and HSIC baselines.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

To gain insight into the mechanisms behind machine learning methods, it is crucial to establish connections among the features describing data points. However, these correlations often exhibit a high-dimensional and strongly nonlinear nature, which makes them challenging to detect using standard methods. This paper exploits the entanglement between intrinsic dimensionality and correlation to propose a metric that quantifies the (potentially nonlinear) correlation between high-dimensional manifolds. We first validate our method on synthetic data in controlled environments, showcasing its advantages and drawbacks compared to existing techniques. Subsequently, we extend our analysis to large-scale applications in neural network representations. Specifically, we focus on latent representations of multimodal data, uncovering clear correlations between paired visual and textual embeddings, whereas existing methods struggle significantly in detecting similarity. Our results indicate the presence of highly nonlinear correlation patterns between latent manifolds.

Problem

• Detecting nonlinear correlations in high-dimensional data.
• Quantifying correlations between multimodal representations.
• Uncovering hidden patterns in neural network embeddings.

Innovation

• Proposes metric for nonlinear correlation detection
• Validates method on synthetic and neural data
• Focuses on multimodal visual-textual embeddings