Schur's Positive-Definite Network: Deep Learning in the SPD cone with structure

2024-06-13

arXiv.org

AI Summary

This work addresses the challenge of estimating symmetric positive definite (SPD) matrices under joint structural constraints—specifically, strict SPDness and element-wise sparsity—which are critical yet difficult to enforce simultaneously in deep learning. We propose the first deep learning module that provably guarantees SPD outputs while supporting flexible structural modeling. Methodologically, we introduce a novel Schur-complement-based parameterization that intrinsically enforces SPDness at the architectural level; integrate Riemannian manifold embedding for SPD geometry awareness; employ differentiable sparse regularization; and enable gradient backpropagation directly on the SPD manifold. The resulting model is fully end-to-end trainable and jointly satisfies both constraints with theoretical guarantees. Evaluated on brain network inference and covariance estimation tasks, our method achieves 100% compliance with SPDness and user-specified sparsity patterns, while significantly outperforming state-of-the-art approaches in accuracy, stability, and generalization.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Estimating matrices in the symmetric positive-definite (SPD) cone is of interest for many applications ranging from computer vision to graph learning. While there exist various convex optimization-based estimators, they remain limited in expressivity due to their model-based approach. The success of deep learning motivates the use of learning-based approaches to estimate SPD matrices with neural networks in a data-driven fashion. However, designing effective neural architectures for SPD learning is challenging, particularly when the task requires additional structural constraints, such as element-wise sparsity. Current approaches either do not ensure that the output meets all desired properties or lack expressivity. In this paper, we introduce SpodNet, a novel and generic learning module that guarantees SPD outputs and supports additional structural constraints. Notably, it solves the challenging task of learning jointly SPD and sparse matrices. Our experiments illustrate the versatility and relevance of SpodNet layers for such applications.

Problem

• Estimating SPD matrices with structural constraints
• Overcoming limitations of model-based SPD estimators
• Ensuring SPD and sparse outputs in neural networks

Innovation

• Deep learning for SPD matrix estimation
• Ensures SPD outputs with structural constraints
• Joint learning of SPD and sparse matrices