CBA: Communication-Bound-Aware Cross-Domain Resource Assignment for Pipeline-Parallel Distributed LLM Training in Dynamic Multi-DC Optical Networks

2025-12-23

AI Summary

To address the cross-domain communication bottleneck hindering large language model (LLM) pipeline parallel training in multi-datacenter optical networks, this paper proposes the first communication-aware cross-domain resource allocation framework. The framework jointly models the strong coupling among dynamic optical network bandwidth, inter-datacenter transmission latency, and computational load, integrating integer linear programming optimization, communication–computation co-modeling, and real-time topology-aware scheduling. Experimental results demonstrate that, compared to baseline approaches, the framework reduces per-iteration training time by 31.25% and request blocking rate by 13.20%, while significantly improving training throughput and heterogeneous resource utilization. It establishes a scalable, optical-network–aware co-optimization paradigm for large-scale distributed LLM training.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We propose a communication-bound-aware cross-domain resource assignment framework for pipeline-parallel distributed training over multi-datacenter optical networks, which lowers iteration time by 31.25% and reduces 13.20% blocking requests compared to baselines.

Problem

• Optimizes cross-domain resource assignment for distributed LLM training
• Reduces communication bottlenecks in multi-datacenter optical networks
• Improves training efficiency by lowering iteration time and blocking requests

Innovation

• Communication-bound-aware cross-domain resource assignment framework
• Pipeline-parallel distributed LLM training optimization
• Dynamic multi-datacenter optical network resource allocation