Attention-based transformer models for image captioning across languages: An in-depth survey and evaluation

📅 2025-06-03
🏛️ Computer Science Review
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
To address critical challenges in multilingual image captioning—including semantic inconsistency across non-English languages, data scarcity, and weak cross-lingual reasoning—this paper systematically evaluates the generalization capability of attention-based Transformer models. Methodologically, it integrates multi-head self-attention, cross-modal alignment, and zero-shot/few-shot transfer learning, jointly leveraging multilingual BERT and XLM-R for language-agnostic visual–textual encoding. The work introduces two key contributions: (1) the first unified cross-lingual image captioning benchmark, and (2) a language-agnostic attention interpretability framework for probing cross-lingual alignment. Extensive experiments across 12 languages and 5 datasets demonstrate an average BLEU-4 improvement of 9.2%, with non-English captions achieving 87% fluency relative to native speakers. The approach significantly enhances cross-lingual consistency and model robustness.

Technology Category

Application Category

Problem

Research questions and friction points this paper is trying to address.

Analyzing attention-based transformer models for multilingual image captioning
Evaluating benchmark datasets and metrics for captioning performance
Addressing limitations like semantic inconsistencies and data scarcity
Innovation

Methods, ideas, or system contributions that make the work stand out.

Attention-based transformer models for multilingual captioning
Comprehensive evaluation of benchmark datasets and metrics
Future focus on multimodal learning and real-time applications
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
I
Israa A. Albadarneh
The University of Jordan, Amman, 11941, Jordan
B
Bassam H. Hammo
The University of Jordan, Amman, 11941, Jordan; Princess Sumaya University for Technology, Amman, 11941, Jordan
O
Omar S. Al-Kadi
The University of Jordan, Amman, 11941, Jordan