Spiky Rank and Its Applications to Rigidity and Circuits

2026-02-26

AI Summary

This work addresses the challenge of effectively characterizing the complexity of matrices that simultaneously exhibit combinatorial and algebraic structure—a key issue in understanding matrix rigidity and circuit lower bounds for neural networks. We introduce a novel parameter, “spiky rank,” which expresses a matrix as a sum of block-diagonal structures whose diagonal blocks are arbitrary rank-one matrices, thereby unifying combinatorial patterns with linear-algebraic flexibility. As a more robust measure of complexity, spiky rank is analyzed through both random matrix theory and explicit constructions, establishing theoretical connections to matrix rigidity, depth-2 ReLU circuit lower bounds, and the γ₂-norm. Our results demonstrate that high spiky rank implies high rigidity and yield tight circuit lower bounds for Hamming distance matrices and spectral expanders.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce spiky rank, a new matrix parameter that enhances blocky rank by combining the combinatorial structure of the latter with linear-algebraic flexibility. A spiky matrix is block-structured with diagonal blocks that are arbitrary rank-one matrices, and the spiky rank of a matrix is the minimum number of such matrices required to express it as a sum. This measure extends blocky rank to real matrices and is more robust for problems with both combinatorial and algebraic character. Our conceptual contribution is as follows: we propose spiky rank as a well-behaved candidate matrix complexity measure and demonstrate its potential through applications. We show that large spiky rank implies high matrix rigidity, and that spiky rank lower bounds yield lower bounds for depth-2 ReLU circuits, the basic building blocks of neural networks. On the technical side, we establish tight bounds for random matrices and develop a framework for explicit lower bounds, applying it to Hamming distance matrices and spectral expanders. Finally, we relate spiky rank to other matrix parameters, including blocky rank, sparsity, and the $\gamma_2$-norm.

Problem

• spiky rank
• matrix rigidity
• ReLU circuits
• matrix complexity
• blocky rank

Innovation

• spiky rank
• matrix rigidity
• ReLU circuits
• blocky rank
• matrix complexity