Topological Blind Spots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity

2024-08-10

arXiv.org

AI Summary

Higher-order message passing (HOMP) in topological deep learning (TDL) lacks expressivity for fundamental topological and metric invariants—such as diameter, orientability, planarity, and homology—and suffers from inadequate lifting and pooling mechanisms on simplicial complexes. Method: We propose Multi-Cell Networks (MCNs) and their scalable variant (SMCNs), the first framework to systematically characterize the expressive boundaries of TDL from a purely topological perspective. MCNs achieve full expressivity over simplicial complex topology, while SMCNs retain strong expressivity with scalability for large-scale training. Contribution/Results: We introduce the first benchmark dataset dedicated to learning complex topological properties. Experiments demonstrate that SMCN significantly outperforms HOMP and powerful graph neural networks on both this new benchmark and real-world graph datasets, validating the efficacy of topology-aware architectural design.

Technology Category

• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Social Networks and Social Media: Computational social science

Abstract

Topological deep learning (TDL) is a rapidly growing field that seeks to leverage topological structure in data and facilitate learning from data supported on topological objects, ranging from molecules to 3D shapes. Most TDL architectures can be unified under the framework of higher-order message-passing (HOMP), which generalizes graph message-passing to higher-order domains. In the first part of the paper, we explore HOMP's expressive power from a topological perspective, demonstrating the framework's inability to capture fundamental topological and metric invariants such as diameter, orientability, planarity, and homology. In addition, we demonstrate HOMP's limitations in fully leveraging lifting and pooling methods on graphs. To the best of our knowledge, this is the first work to study the expressivity of TDL from a emph{topological} perspective. In the second part of the paper, we develop two new classes of architectures -- multi-cellular networks (MCN) and scalable MCN (SMCN) -- which draw inspiration from expressive GNNs. MCN can reach full expressivity, but scaling it to large data objects can be computationally expansive. Designed as a more scalable alternative, SMCN still mitigates many of HOMP's expressivity limitations. Finally, we create new benchmarks for evaluating models based on their ability to learn topological properties of complexes. We then evaluate SMCN on these benchmarks and on real-world graph datasets, demonstrating improvements over both HOMP baselines and expressive graph methods, highlighting the value of expressively leveraging topological information. Code and data are available at https://github.com/yoavgelberg/SMCN.

Problem

• Analyzes HOMP's limitations in capturing topological invariants.
• Proposes MCN and SMCN for enhanced topological expressivity.
• Introduces benchmarks for evaluating topological learning models.

Innovation

• Explores higher-order message-passing expressivity
• Introduces multi-cellular networks architecture
• Develops scalable multi-cellular networks alternative