FeLoG: Scalable and Efficient Distributed Graph Embedding with Feedback Loop Mechanism

2026-06-20

AI Summary

This work addresses the inefficiencies in large-scale graph embedding caused by the decoupling of sampling and training, which leads to redundant computation, high communication overhead, and poor resource utilization. To bridge this gap, the authors propose a feedback-driven distributed graph embedding system that tightly integrates sampling and training through dynamic priority-based sampling guided by real-time feedback, node sequence compression, selective embedding synchronization, and a CPU-GPU interleaved pipelining strategy. Evaluated against six state-of-the-art baselines, the system achieves an average speedup of 27.9×, reduces communication overhead by over 53.1%, and maintains CPU-GPU utilization above 80%, substantially enhancing both training efficiency and scalability.

Abstract

Graph embedding maps graph nodes into low-dimensional vectors to support applications such as recommendation, fraud detection, and graph-based retrieval-augmented generation (GraphRAG). As graphs scale to billions of edges, scalable and efficient graph embedding has become increasingly important. Existing frameworks commonly adopt a sampling-training paradigm, in which mini-batches are constructed by sampling nodes and their neighbors. However, sampling is typically decoupled from evolving embedding quality, causing redundant exploration of well-trained regions while under-sampling undertrained nodes. At the system level, such decoupling further leads to excessive communication, serialized execution, and low resource utilization in distributed environments. We present FeLoG, a feedback loop-driven system for scalable distributed graph embedding. (1) FeLoG introduces feedback-coupled sampling and training, dynamically prioritizing undertrained nodes according to real-time embedding-quality feedback, thereby reducing redundant computation and accelerating convergence. (2) It employs activity-aware communication that compresses frequently occurring node sequences to reduce intra-machine PCIe traffic and selectively synchronizes frequently updated embeddings to reduce inter-machine communication. (3) It adopts a round-interleaved pipeline that overlaps next-round sampling with current-round training to improve CPU-GPU utilization. Experiments against six state-of-the-art baselines on large-scale graphs show that FeLoG achieves an average speedup of 27.9x, reduces communication cost by more than 53.1%, and sustains over 80% CPU-GPU utilization.

Problem

• graph embedding
• scalable distributed system
• sampling efficiency
• communication overhead
• resource utilization

Innovation

• feedback loop
• distributed graph embedding
• activity-aware communication
• round-interleaved pipeline
• scalable graph learning