Optimal Expert Selection for Distributed Mixture-of-Experts at the Wireless Edge

2025-03-17

AI Summary

In 6G edge AI, existing distributed Mixture-of-Experts (MoE) systems suffer from suboptimal joint optimization of expert selection and wireless channel allocation—failing to simultaneously capture task-expert semantic relevance and dynamic channel diversity—leading to excessive communication overhead and degraded inference accuracy. Method: This paper proposes a Joint Expert Selection and Subcarrier Allocation (JESA) framework. It is the first to unify expert selection with OFDMA subcarrier assignment in a single optimization model, designing a Dynamic Expert Selection (DES) algorithm with an asymptotically optimal iterative solver. A tunable importance factor enables flexible accuracy–overhead trade-offs. Linear relaxation-based pruning and approximation techniques address the underlying NP-hardness. Contribution/Results: JESA significantly improves inference accuracy while reducing measured communication overhead by 42%. It demonstrates robustness and real-time adaptability in heterogeneous edge environments.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

The emergence of distributed Mixture-of-Experts (DMoE) systems, which deploy expert models at edge nodes, offers a pathway to achieving connected intelligence in sixth-generation (6G) mobile networks and edge artificial intelligence (AI). However, current DMoE systems lack an effective expert selection algorithm to address the simultaneous task-expert relevance and channel diversity inherent in these systems. Traditional AI or communication systems focus on either performance or channel conditions, and direct application of these methods leads to high communication overhead or low performance. To address this, we propose the DMoE protocol to schedule the expert inference and inter-expert transmission. This protocol identifies expert selection and subcarrier allocation as key optimization problems. We formulate an expert selection problem by incorporating both AI performance and channel conditions, and further extend it to a Joint Expert and Subcarrier Allocation (JESA) problem for comprehensive AI and channel management within the DMoE framework. For the NP-hard expert selection problem, we introduce the Dynamic Expert Selection (DES) algorithm, which leverages a linear relaxation as a bounding criterion to significantly reduce search complexity. For the JESA problem, we discover a unique structural property that ensures asymptotic optimality in most scenarios. We propose an iterative algorithm that addresses subcarrier allocation as a subproblem and integrates it with the DES algorithm. The proposed framework effectively manages the tradeoff between task relevance and channel conditions through a tunable importance factor, enabling flexible adaptation to diverse scenarios. Numerical experiments validate the dual benefits of the proposed expert selection algorithm: high performance and significantly reduced cost.

Problem

• Lack of effective expert selection in DMoE systems
• High communication overhead or low performance in current methods
• Need for joint AI performance and channel condition optimization

Innovation

• DMoE protocol schedules expert inference and transmission
• Dynamic Expert Selection algorithm reduces search complexity
• Joint Expert and Subcarrier Allocation ensures asymptotic optimality