HybridRAG-BN: A Retrieval-Augmented Framework with Fine-Tuned Verification for Bangla KBQA

2026-08-13

AI Summary

This work addresses the challenges of weak retrieval capability, scarce linguistic resources, and difficulty in accurately grounding answers to external knowledge in Bengali knowledge base question answering (KBQA). To tackle these issues, we propose the first end-to-end framework integrating hybrid retrieval, large language model generation, and lightweight fine-tuning for verification. Our approach combines BM25 with BGE-M3 for hybrid retrieval, leverages Gemma-4-31B-Instruct (in GGUF format) for answer generation, and employs LoRA fine-tuning to enhance generation accuracy, further augmented by DuckDuckGo fallback search for robustness. The system achieves token-level F1 scores of 0.71654 and 0.72912 on public and private test sets, respectively—both ranking first in the competition—and substantially advances the performance frontier for KBQA in low-resource languages.

Abstract

Knowledge-base question answering (KBQA) systems rely on effective retrieval and reasoning mechanisms to generate accurate answers from external knowledge sources. However, developing reliable KBQA systems for low-resource languages such as Bangla remains challenging due to limited retrieval-focused research, scarce language resources, and difficulties in grounding generated responses in external knowledge. In this work, we propose HybridRAG-BN, a retrieval-augmented framework for Bangla KBQA that integrates hybrid retrieval using BM25 and BGE-M3, answer generation using the GGUF version of Gemma-4-31B-Instruct, and a LoRA-fine-tuned Gemma-4-31B-Instruct model for answer verification and refinement. To further improve robustness, the framework incorporates a post-processing stage that addresses unresolved cases through fallback answer replacement and DuckDuckGo-assisted retrieval. Experimental results demonstrate the effectiveness of the proposed framework, achieving token-level F1 scores of 0.71654 and 0.72912 on the public and private leaderboards, respectively, securing first place in the competition.

Problem

• KBQA
• low-resource languages
• Bangla
• retrieval
• knowledge grounding

Innovation

• Hybrid Retrieval
• LoRA Fine-tuning
• Retrieval-Augmented Generation
• Low-resource KBQA
• Answer Verification