CoRDE: Concept-Prior Routed Diffusion Experts for Structural Generalization in Robot Manipulation

2026-06-20

AI Summary

This work addresses the limitations of existing mixture-of-experts diffusion models in multitask and long-horizon robotic manipulation, which suffer from insufficient structural generalization due to gradient conflicts, routing collapse, and parameter inflation. To overcome these challenges, the authors propose a structure-guided variational distillation framework that leverages a frozen concept encoder to extract semantic priors and integrates an entropy-controlled responsibility inference mechanism with a learnable soft mapping matrix to achieve semantically aligned expert specialization. The approach further incorporates a low-rank efficient expert pool based on LoRA alongside a frozen backbone network, substantially reducing parameter overhead while preventing routing collapse. Theoretical analysis provides an upper bound on mixture score error, and experiments demonstrate superior performance over current baselines in both action quality and incremental learning efficiency.

Abstract

Diffusion models excel at capturing multi-modal action distributions in robot imitation learning. However, in multi-task and long-horizon scenarios, monolithic architectures lack structural generalization capabilities, suffering from gradient conflicts between distinct semantic sub-stages. While pure data-driven Mixture-of-Experts (MoE) methods introduce labor division, they frequently trigger routing collapse, and instantiating full-scale experts causes parameter explosion and high expansion costs. To address these issues, we propose Concept-prior Routed Diffusion Experts (CoRDE), a structure-guided variational distillation framework. CoRDE extracts semantic distributions from a frozen concept encoder to guide the variational posterior responsibility via a learnable soft mapping matrix. This mechanism introduces an entropy-controlled responsibility inference process that encourages confident routing under reliable semantic predictions while preserving the stochastic diffusion term for behavioral diversity. To overcome parameter inflation, CoRDE employs a parameter-efficient expert pool using Low-Rank Adaptation (LoRA) on a shared frozen backbone. Theoretical analysis shows that the mixture score discrepancy is bounded by responsibility-weighted local expert errors, supporting high-fidelity generation under low-rank expert adaptation. Empirical evaluations confirm that, compared to existing baselines, CoRDE systematically reduces routing collapse, forming robust, semantically aligned expert allocations while achieving superior action quality and incremental learning efficiency.

Problem

• structural generalization
• diffusion models
• Mixture-of-Experts
• routing collapse
• parameter explosion

Innovation

• Diffusion Models
• Mixture-of-Experts
• Structural Generalization
• Low-Rank Adaptation
• Semantic Routing