Near optimal edge partitioning via intersecting families

2025-05-23

AI Summary

This paper studies the graph edge partitioning problem: partitioning the edges of a graph into $k$ nearly equal-sized subsets to minimize the vertex replication factor—the number of vertices assigned to multiple partitions. To overcome the limited expressiveness of conventional symmetric intersecting families, we introduce the “balanced intersecting system,” a novel combinatorial structure that relaxes symmetry constraints and improves adaptability to arbitrary $k$. Leveraging tools from combinatorial design, intersecting family theory, and asymptotic analysis, we construct an edge partition achieving a replication factor of $sqrt{n}(1+o(1))$, matching the theoretical lower bound for this problem. Our method ensures near-perfect load balancing across partitions and is universally applicable to any $k$, thereby significantly advancing both the theoretical foundations and practical applicability of graph partitioning in distributed graph processing.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Combinatorial Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We study the problem of edge-centric graph partitioning, where the goal is to distribute the edges of a graph among several almost equally sized partitions in order to minimize the replication factor of vertices. We build a partitioning algorithm that guarantees near-perfect balance and replication factor $sqrt{n}(1 + o(1))$ for arbitrary number of partitions $n$. This asymptotical bound cannot be improved. To do so, we introduce balanced intersecting systems. It is a construction similar to symmetric intersecting families, but the symmetry condition is replaced by a weaker balance condition. We build an algorithm that uses such a system, and prove that by using a system of optimal cardinality we achieve exactly optimal guarantees for the replication factor. Finally, we build balanced intersecting systems with asymptotically optimal cardinality.

Problem

• Edge-centric graph partitioning for balanced edge distribution
• Minimizing vertex replication factor in graph partitions
• Constructing balanced intersecting systems for optimal partitioning

Innovation

• Edge-centric graph partitioning algorithm
• Balanced intersecting systems construction
• Asymptotically optimal cardinality systems