Statistic-Augmented, Decoupled MoE Routing and Aggregating in Autonomous Driving

2025-12-07

AI Summary

To address the limited generalization of single models under varying weather conditions, traffic densities, and road types in autonomous driving, this paper proposes a large-model-driven disentangled Mixture-of-Experts (MoE) system. We introduce a statistically enhanced expert routing mechanism that leverages prototype caching and statistical distance metrics to improve routing accuracy. Furthermore, we propose an adaptive reweighting aggregation strategy that decouples feature matching from output fusion, enhancing ensemble efficiency. Our method integrates large-model-based feature extraction with dynamic MoE routing and aggregation for semantic segmentation. Experiments on multiple autonomous driving benchmarks demonstrate significant improvements over both single-model baselines and state-of-the-art MoE approaches, validating superior robustness and adaptability in complex, real-world driving scenarios.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Large Vision Models
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Autonomous driving (AD) scenarios are inherently complex and diverse, posing significant challenges for a single deep learning model to effectively cover all possible conditions, such as varying weather, traffic densities, and road types. Large Model (LM)-Driven Mixture of Experts (MoE) paradigm offers a promising solution, where LM serves as the backbone to extract latent features while MoE serves as the downstream head to dynamically select and aggregate specialized experts to adapt to different scenarios. However, routing and aggregating in MoE face intrinsic challenges, including imprecise expert selection due to flawed routing strategy and inefficient expert aggregation leading to suboptimal prediction. To address these issues, we propose a statistic-augmented, decoupled MoE }outing and Aggregating Mechanism (MoE-RAM) driven by LM. Specifically, on the one hand, MoE-RAM enhances expert routing by incorporating statistical retrieval mechanism to match LM-extracted latent features with cached prototypical features of the most relevant experts; on the other hand, MoE-RAM adaptively reweights experts' outputs in fusion by measuring statistical distances of experts' instant features against LM-extracted latent features. Benefiting from the synergy of the statistic-augmented MoE's routing and aggregating, MoE-RAM ultimately improves the prediction performance. We take the AD semantic segmentation task as an example to assess the proposed MoE-RAM. Extensive experiments on AD datasets demonstrate the superiority of MoE-RAM compared to other MoE baselines and conventional single-model approaches.

Problem

• Enhances expert selection in MoE via statistical retrieval for better routing
• Improves expert aggregation by adaptively reweighting outputs based on statistical distances
• Boosts prediction performance in autonomous driving semantic segmentation tasks

Innovation

• Statistic-augmented routing matches features with cached prototypes
• Decoupled mechanism reweights experts by measuring statistical distances
• LM-driven MoE improves prediction via enhanced routing and aggregation