Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering

2026-05-26

AI Summary

This work addresses the insufficient cross-lingual stability of large language models when handling culturally grounded question answering in low-resource languages. To mitigate this issue, the authors propose a region-aware hybrid retrieval approach that combines BM25 lexical matching with dense semantic retrieval, augmented by a region-specific cultural weighting heuristic. This strategy constructs structured prompts for the quantized Qwen3-14B model, enabling deterministic answer selection based on logit scores. Evaluated on the BLEnD multilingual benchmark, the method significantly improves both relevance and cross-lingual consistency in cultural commonsense question answering. The results demonstrate the advantages of hybrid retrieval over purely parametric reasoning while also highlighting persistent performance gaps between high- and low-resource languages.

Abstract

Although Large Language Models (LLMs) demonstrate excellent capabilities and performance for general reasoning tasks within the general public domain, they may face challenges with culturally grounded knowledge within languages with limited digital and textual data. In this paper, we investigate culturally grounded multiple-choice question answering with the BLEnD benchmark, which consists of a multilingual corpus of 30 languages and covers various socio-cultural domains, such as cuisine, sports, family, etc. We propose a region-aware hybrid retrieval approach that combines BM25 lexical matching and dense semantic similarity with regional weighting heuristics to improve the relevance of the answer. The retrieved documents are used to construct a structured prompt for the Qwen3-14B quantized model with logit-based deterministic answer selection. The experimental results show improvements to cross-lingual stability with the hybrid retrieval approach over pure parametric inference for culturally grounded question answering. However, there are still notable performance gaps between languages with more and less training data. This shows that the limitations of the retrieval augmentation approach are not entirely overcome by the training data imbalance problem.

Problem

• low-resource languages
• cultural reasoning
• multilingual question answering
• training data imbalance
• cross-lingual stability

Innovation

• region-aware retrieval
• hybrid retrieval
• cultural reasoning
• low-resource multilingual QA
• retrieval-augmented LLM