Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry

2024-07-15

arXiv.org

AI Summary

Existing studies fail to explain why Euclidean classifiers can be directly applied to Riemannian features after matrix power normalization. Method: From a Riemannian geometric perspective, we provide the first unified interpretation of the intrinsic roles of matrix logarithm and power normalization on the Symmetric Positive Definite (SPD) manifold: they are not mere linear mappings but implicitly realize a Riemannian classifier—equivalently performing geodesic distance classification on the manifold via tangent space projection. We establish a rigorous theoretical correspondence between normalization operations and the Riemannian classifier, grounded in covariance pooling modeling, matrix function analysis, and SPD manifold theory. Contribution/Results: Our unified framework ensures theoretical consistency, significantly enhancing both interpretability and performance. Extensive experiments on fine-grained and large-scale visual classification benchmarks validate the theory. The implementation is publicly available.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Representation Learning for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations. GCP typically performs classification of the covariance matrices by applying matrix function normalization, such as matrix logarithm or power, followed by a Euclidean classifier. However, covariance matrices inherently lie in a Riemannian manifold, known as the Symmetric Positive Definite (SPD) manifold. The current literature does not provide a satisfactory explanation of why Euclidean classifiers can be applied directly to Riemannian features after the normalization of the matrix power. To mitigate this gap, this paper provides a comprehensive and unified understanding of the matrix logarithm and power from a Riemannian geometry perspective. The underlying mechanism of matrix functions in GCP is interpreted from two perspectives: one based on tangent classifiers (Euclidean classifiers on the tangent space) and the other based on Riemannian classifiers. Via theoretical analysis and empirical validation through extensive experiments on fine-grained and large-scale visual classification datasets, we conclude that the working mechanism of the matrix functions should be attributed to the Riemannian classifiers they implicitly respect. The code is available at https://github.com/GitZH-Chen/RiemGCP.git.

Problem

• Explains matrix function normalizations in GCP.
• Analyzes Euclidean classifiers on Riemannian manifolds.
• Validates matrix functions' mechanisms in visual classification.

Innovation

• Explores Riemannian geometry in GCP
• Analyzes matrix logarithm and power
• Validates through extensive classification datasets