TopoBenchmarkX: A Framework for Benchmarking Topological Deep Learning

2024-06-09

arXiv.org

AI Summary

The topological deep learning (TDL) field lacks a unified benchmarking framework, hindering model comparability and research reproducibility. Method: We propose TDL-Bench—the first modular, open-source benchmarking framework for TDL. It decouples the full pipeline into data loading, topological domain transformations (e.g., graphs ↔ simplicial complexes ↔ cell complexes), feature lifting, model training, and evaluation. Crucially, it introduces a novel operator library supporting reversible mappings between arbitrary topological domains and high-order structural embeddings. It integrates simplicial and cell complex embedding modules alongside a standardized evaluation protocol. Results: We systematically evaluate mainstream TDL models across multiple datasets. Experiments demonstrate that TDL-Bench significantly enhances representation granularity and analytical flexibility, accelerating TDL algorithm development, validation, and deployment. By establishing a standardized, extensible benchmarking paradigm, TDL-Bench lays the foundation for rigorous, reproducible progress in topological deep learning.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This work introduces TopoBenchmarkX, a modular open-source library designed to standardize benchmarking and accelerate research in Topological Deep Learning (TDL). TopoBenchmarkX maps the TDL pipeline into a sequence of independent and modular components for data loading and processing, as well as model training, optimization, and evaluation. This modular organization provides flexibility for modifications and facilitates the adaptation and optimization of various TDL pipelines. A key feature of TopoBenchmarkX is that it allows for the transformation and lifting between topological domains. This enables, for example, to obtain richer data representations and more fine-grained analyses by mapping the topology and features of a graph to higher-order topological domains such as simplicial and cell complexes. The range of applicability of TopoBenchmarkX is demonstrated by benchmarking several TDL architectures for various tasks and datasets.

Problem

• Standardize benchmarking in topological deep learning
• Enable flexible modular TDL pipeline optimization
• Support transformations across topological domains

Innovation

• Modular framework for topological deep learning
• Supports transformations across topological domains
• Benchmarks diverse TDL architectures effectively