Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation
This study addresses the controversy regarding the correlation between routing divergence and behavioral impact in equal-weight MoE self-distillation. We propose a block-level decomposition method to disentangle routing from content components, complemented by residual stream exposure analysis and noise-controlled experiments. Results demonstrate that routing shifts do not equate to behavioral changes; their effects are significantly weaker than those induced by natural contexts and can be replicated by matched noise. Consequently, this work reveals the non-behavioral nature of routing divergence and introduces an innovative "exposure-first" evaluation mechanism. By advocating for exposure assessment prior to intervention, we establish a novel paradigm for interpreting MoE internal mechanisms and optimizing self-distillation processes.