CoGR-MoE: Concept-Guided Expert Routing with Consistent Selection and Flexible Reasoning for Visual Question Answering

2026-04-18

AI Summary

This work addresses the challenge in visual question answering where expert routing often suffers from either excessive stability or instability, leading to inconsistent reasoning for similar questions or insufficient flexibility. To mitigate this, the authors propose a concept-guided expert routing framework that leverages semantic information from answer choices during training to guide expert selection. Specifically, option features are re-weighted to generate discriminative representations that facilitate both option-wise comparison and contrastive learning. By integrating mixture-of-experts (MoE) architecture, semantic-guided routing, and contrastive optimization, the method enhances reasoning flexibility while preserving routing consistency. Extensive experiments demonstrate that the proposed framework significantly outperforms existing approaches across multiple visual question answering benchmarks, confirming its effectiveness and strong generalization capability.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Visual Reasoning & Symbolic Representations
• Natural Language Processing: Question Answering

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Community question answering
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Visual Question Answering (VQA) requires models to identify the correct answer options based on both visual and textual evidence. Recent Mixture-of-Experts (MoE) methods improve option reasoning by grouping similar concepts or routing based on examples. However, unstable routing can lead to inconsistent expert selection in the same question type, while overly stable routing may reduce flexibility. To address this, we propose Concept-Guided Routing framework (CoGR-MoE), which incorporates semantics of the answer options to guide expert selection in the training phase. Next, option features are used to reweight the selected experts, producing discriminative representations for each candidate option. These option-level representations are further used for option comparison and optimized via contrastive learning. The experimental results indicate that CoGR-MoE delivers strong performance across multiple VQA tasks, demonstrating the effectiveness of our approach.

Problem

• Visual Question Answering
• Mixture-of-Experts
• Expert Routing
• Consistency
• Flexibility

Innovation

• Concept-Guided Routing
• Mixture-of-Experts
• Visual Question Answering
• Contrastive Learning
• Expert Selection