memory-efficient algorithms

Efficient storage and updating of large-scale dynamic graphs face the challenge of balancing query performance against memory overhead. This work proposes a vertex indexing scheme based on a space-optimal pointer-array radix tree, integrated with a snapshot-log hybrid edge storage architecture, achieving amortized O(1) update complexity and strong support for high concurrency. The approach maintains high performance for graph analytics while significantly outperforming existing methods: it improves update throughput by up to 16.27× and reduces memory consumption by 40.1% on average.

Dynamic graph processing faces a fundamental trade-off between computational efficiency—requiring contiguous memory layouts—and update efficiency—necessitating pointer-based, mutable structures—while cache misses dominate performance bottlenecks. To address this, we propose CBList, a prefetch-friendly dynamic graph storage structure, and pioneer the integration of stackless coroutines into graph traversal to enable fine-grained, low-overhead software prefetching. By tightly co-designing CBList’s memory layout with coroutine-driven prefetching, our approach simultaneously achieves high spatial locality and structural mutability. Experimental evaluation demonstrates 1.3×–180× speedup in graph updates and 1.4×–41.1× speedup in graph computation over state-of-the-art dynamic graph systems. Our core contribution is the first systematic co-design paradigm unifying storage structure and coroutine-based prefetching, establishing a novel, cache-aware methodology for dynamic graph processing.

This work addresses classical parameterized graph problems—including $k$-Path, MaxLeaf SubTree, and Tree Multicut—under memory-constrained settings. Moving beyond conventional vertex/edge deletion paradigms, we introduce a novel graph structural compression scheme coupled with lightweight tree decomposition traversal. Our approach integrates controlled recursion depth with polylogarithmic-space dynamic programming. This yields the first unified framework achieving $f(k) cdot mathrm{poly}(n)$ time and $g(k) cdot mathrm{polylog}(n)$ space complexity for these problems. Crucially, our algorithms avoid $Omega(mathrm{poly}(n))$ memory overhead, enabling scalability from gigabyte- to terabyte-scale graphs. The proposed methods significantly alleviate memory bottlenecks in large-scale graph processing and constitute the first systematic sublinear-space solution for parameterized graph computation in memory-sensitive environments.

This paper addresses performance bottlenecks of in-memory dynamic graph storage (DGS) under high-concurrency read/write workloads, identifying three fundamental issues: (1) excessive memory redundancy—3.3× to 8.9× higher than CSR; (2) poor cache efficiency due to neglect of modern memory access patterns; and (3) severe contention and versioning overhead on high-degree vertices caused by fine-grained concurrency control. To systematically analyze these challenges, the authors propose a unified DGS abstraction model and a configurable, multi-dimensional benchmarking framework—the first to quantitatively evaluate trade-offs among throughput, latency, memory footprint, and cache behavior across mainstream DGS designs. Empirical results demonstrate that fine-grained versioning is fundamentally unsuitable for highly concurrent dynamic graph workloads. The study provides both theoretical foundations and practical guidance for designing next-generation DGS architectures that are low-overhead, cache-friendly, and scalable.

Traditional graph processing systems are constrained by monolithic architectures, where tightly coupled resources lead to low utilization. Existing memory-disaggregated approaches suffer from poor scalability and high cache overhead. This work proposes DMG, the first practical memory-disaggregated graph processing system, which introduces a disaggregation-friendly graph storage layout, an adaptive update coordination mechanism, and a two-level load management strategy to enable efficient graph access, low-overhead update propagation, and dynamic load balancing. DMG is the first system to support elastic scaling across multiple compute and memory nodes while significantly reducing cache requirements without sacrificing performance. Experimental results demonstrate that DMG achieves up to 4.9× higher performance and reduces cache footprint by up to 18.9× compared to the state-of-the-art systems.

This work addresses the memory wall bottleneck in large-scale graph processing, which stems from frequent data movement between processors and memory. For the first time, it presents a systematic evaluation of representative graph algorithms on a commercial Processing-in-Memory (PIM) platform—UPMEM—leveraging an algebraic formulation to guide optimizations. By integrating DMA-based data transfers, multi-core parallelism, and tailored data partitioning strategies, the approach substantially reduces data movement overhead. Experimental comparisons against CPU and GPU baselines uncover critical limitations in current PIM architectures concerning computation throughput, communication efficiency, and memory subsystem design. The study further identifies key optimization directions, including enhancing instruction-level parallelism, enabling non-blocking DMA operations, and incorporating direct interconnects among PIM cores, thereby offering valuable insights for future PIM architecture development.

Large-scale graph data often exceeds main-memory capacity, while existing out-of-core graph processing systems suffer from inefficient I/O (i.e., high read and work amplification) and severe synchronization stalls due to rigidly synchronized iterations, leading to underutilized SSDs. To address these challenges, this paper proposes AsyncGraph—a novel asynchronous out-of-core graph processing framework designed for SSDs. Its core contributions are: (1) a workload-aware dynamic block-level priority scheduler coupled with an online asynchronous worklist, significantly reducing redundant disk accesses; and (2) deep pipelining of computation and asynchronous I/O, enhanced by a hybrid storage format (optimized for low-degree vertex access) and an active-block in-memory reuse mechanism, thereby sustaining high SSD throughput. Evaluated on BFS, WCC, and PageRank, AsyncGraph achieves an average 2.3× speedup and 41% higher I/O efficiency over state-of-the-art systems.

This work addresses the memory bandwidth bottleneck in billion-scale graph-based approximate nearest neighbor search (ANNS), where existing processing-in-memory (PIM) approaches struggle to achieve both high recall and high throughput due to limited memory capacity, substantial communication overhead, and weak computational capability. Through algorithm-architecture co-design, the authors introduce a compact index layout that reduces memory footprint by 14.5×, an asynchronous pipelined scheduler that keeps the host-PIM interconnect continuously saturated, and a multiplication-free distance kernel that incurs less than 0.08% recall loss. For the first time, this enables PIM-based ANNS to approach the theoretical recall limits of graph methods. Evaluated on three billion-scale benchmarks, the system achieves 20× and 17.1× higher throughput than CPU and GPU baselines, respectively, and outperforms prior PIM solutions by up to 129× under high-recall settings, while supporting seamless scaling across multi-node systems and emerging PIM architectures.

This work addresses the fundamental trade-off between update efficiency and traversal locality in in-memory dynamic graph systems under highly skewed degree distributions. To reconcile this tension, the authors propose a degree-aware hierarchical graph storage architecture that, for the first time, integrates learned indexes into graph data management. By decoupling vertex and edge access and adaptively organizing edge data using either lightweight arrays or learned indexes based on vertex degree, the design achieves both high update throughput and efficient graph traversals. Combined with a degree-aware memory layout and multicore-friendly memory management, the system optimizes performance for both frequent updates and complex analytical queries within a unified framework. Experimental results demonstrate 5.9–28.2× higher update throughput and significantly accelerated graph analysis compared to state-of-the-art in-memory graph systems.

This work addresses the challenges of high startup latency and low query efficiency in graph analytics under lakehouse architectures. The authors propose a lakehouse-native graph processing engine that maps lakehouse tables to vertex and edge types in a property graph and enables efficient querying through GSQL. Key innovations include loading only graph topology to accelerate system initialization, designing a graph-aware caching mechanism, and developing two lakehouse-optimized parallel primitives for graph computation. Experimental evaluation demonstrates that the proposed system significantly outperforms PuppyGraph—the current state-of-the-art—in both startup time and query latency across a range of workloads.