Problem
Research questions and friction points this paper is trying to address.
Large Language Models
Bangla Idioms
Idiomatic Expressions
Low-Resource Languages
To What Extent Do Large Language Models Understand Bangla Idioms?
🤖 AI Summary
本文构建了首个大规模孟加拉语成语基准数据集，并通过三项任务评估了大型语言模型对成语的理解能力，使用零样本和少样本提示策略。
📝 Abstract
Idiomatic expressions are an integral part of natural language, reflecting cultural nuances and posing unique challenges for computational models, particularly in low-resource languages. In this paper, we present the first large-scale benchmark dataset of Bangla idioms, complemented by a synthetic multiple-choice question (MCQ) dataset for idiom meaning identification. We conduct a comprehensive evaluation of recent large language models (LLMs) across three idiom-related tasks: paraphrasing, idiom span detection, and meaning identification, leveraging zero-shot and few-shot prompting strategies. Our results reveal substantial variability in model performance, with no single LLM consistently outperforming others across all tasks. Notably, Phi-4-mini-instruct excels in paraphrasing, Kimi-K2-32b-instruct in span detection, and Gemini-2.5-flash in meaning identification. We believe that our datasets and analyses will provide valuable resources to guide future research in improving LLM comprehension of idiomatic expressions, particularly in Bangla and other low-resource languages.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Large-scale benchmark dataset
Bangla idioms
Synthetic MCQ dataset
Zero-shot and few-shot prompting
Idiomatic expression comprehension
🔎 Similar Papers
No similar papers found.
