Universal and Asymptotically Optimal Data and Task Allocation in Distributed Computing
This work addresses the joint optimization of communication and computation overheads in distributed computing, where a master node coordinates \(N\) workers to compute a set of subfunctions dependent on \(d\) input files. The problem is modeled as a \(d\)-uniform hypergraph edge partitioning task, and a deterministic Interweaved-Cliques (IC) assignment scheme is proposed. This scheme achieves order-optimal communication load (number of files received per worker) and computation load (number of subfunctions processed per worker) without prior knowledge of the subfunction structure. Leveraging an information-theoretically inspired interwoven clique construction and a deterministic allocation strategy, the method applies to any multi-function decomposition satisfying mild density conditions, requires no file reassignment, and attains order-optimal communication cost \(\Theta(n/N^{1/d})\) and computation cost across a broad range of parameters, yielding a partitioning gain of \(N^{1/d}\).