GTX: A Write-Optimized Latch-free Graph Data System with Transactional Support -- Extended Version

2024-05-02

AI Summary

Existing in-memory graph data systems suffer from limited temporal locality in update-intensive workloads due to vertex-level lock contention when supporting ACID transactions and concurrent read-write operations. This paper proposes GTX, a write-optimized, memory-centric graph system. Its core contributions are: (1) an adaptive delta-chain locking protocol that eliminates vertex-level lock contention; (2) a lock-free graph storage design synergistically integrated with block-level multi-version concurrency control (MVCC), ensuring both cache efficiency and robust concurrency management; and (3) a coordinated group commit mechanism coupled with incremental garbage collection. Under write-heavy workloads, GTX achieves up to 11× higher throughput than state-of-the-art systems. It maintains low-latency graph analytics under mixed read-write workloads while preserving the efficiency of read-intensive analytical queries.

Technology Category

• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Security and Privacy: Data transparency and provenance

Abstract

This paper introduces GTX, a standalone main-memory write-optimized graph data system that specializes in structural and graph property updates while enabling concurrent reads and graph analytics through ACID transactions. Recent graph systems target concurrent read and write support while guaranteeing transaction semantics. However, their performance suffers from updates with real-world temporal locality over the same vertices and edges due to vertex-centric lock contentions. GTX has an adaptive delta-chain locking protocol on top of a carefully designed latch-free graph storage. It eliminates vertex-level locking contention, and adapts to real-life workloads while maintaining sequential access to the graph's adjacency lists storage. GTX's transactions further support cache-friendly block level concurrency control, and cooperative group commit and garbage collection. This combination of features ensures high update throughput and provides low-latency graph analytics. Based on experimental evaluation, in addition to not sacrificing the performance of read-heavy analytical workloads, and having competitive performance similar to state-of-the-art systems, GTX has high read-write transaction throughput. For write-heavy transactional workloads, GTX achieves up to 11x better transaction throughput than the best-performing state-of-the-art system.

Problem

• Optimizes graph data system for write-heavy workloads.
• Eliminates vertex-level locking contention in graph updates.
• Supports concurrent reads and writes with ACID transactions.

Innovation

• Latch-free graph storage
• Adaptive delta-chain locking
• Cache-friendly block concurrency