A Parallel and Distributed Rust Library for Core Decomposition on Large Graphs

2025-11-28

AI Summary

To address performance bottlenecks in large-scale graph k-core decomposition on modern multicore systems, this paper proposes FastK—a high-performance parallel algorithm tailored for shared-memory architectures. Unlike sequential algorithms or naively ported distributed approaches, FastK redesigns synchronization mechanisms, optimizes inter-thread communication paths, and employs fine-grained concurrency control, all implemented in Rust to ensure memory safety and thread safety. Methodologically, it introduces a three-tier progressive optimization framework: SequentialK → ParallelK → FastK, systematically reducing synchronization overhead. Experimental evaluation on real-world graphs demonstrates that FastK achieves up to 11× speedup over baseline parallel implementations on 16 threads and outperforms NetworkX by approximately two orders of magnitude. FastK thus provides a scalable, high-throughput solution for identifying dense substructures and assessing node centrality in complex networks.

Technology Category

• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Machine Learning: Graph-based Machine Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

In this paper, we investigate the parallelization of $k$-core decomposition, a method used in graph analysis to identify cohesive substructures and assess node centrality. Although efficient sequential algorithms exist for this task, the scale of modern networks requires faster, multicore-ready approaches. To this end, we adapt a distributed $k$-core algorithm originally proposed by Montresor et al. to shared-memory systems and implement it in Rust, leveraging the language's strengths in concurrency and memory safety. We developed three progressively optimized versions: SequentialK as a baseline, ParallelK introducing multi-threaded message passing, and FastK further reducing synchronization overhead. Extensive experiments on real-world datasets, including road networks, web graphs, and social networks, show that FastK consistently outperforms both SequentialK and ParallelK, as well as a reference Python implementation available in the NetworkX library. Results indicate up to an 11x speedup on 16 threads and execution times up to two orders of magnitude faster than the Python implementation.

Problem

• Parallelizing k-core decomposition for large-scale graph analysis.
• Adapting distributed algorithms to shared-memory systems using Rust.
• Optimizing performance to outperform sequential and Python implementations.

Innovation

• Parallel distributed Rust library for k-core decomposition
• Three optimized versions with reduced synchronization overhead
• Achieves up to 11x speedup on 16 threads