Multi-Head Attention based interaction-aware architecture for Bangla Handwritten Character Recognition: Introducing a Primary Dataset

2026-04-08

AI Summary

This study addresses the challenges of Bangla handwritten character recognition, which stem from diverse writing styles, high inter-class similarity, and class imbalance in existing datasets. To overcome these issues, the authors construct a large-scale balanced dataset comprising 78 classes with approximately 650 samples per class. They propose a parallel hybrid architecture that integrates EfficientNetB3, Vision Transformer, and Conformer modules, enabling inter-module feature interaction through a multi-head cross-attention mechanism. This design effectively combines the local perceptual strengths of CNNs, the global modeling capacity of Transformers, and the sequential structure awareness of Conformers. The proposed method achieves state-of-the-art performance with 98.84% accuracy on the newly introduced dataset and 96.49% on the CHBCR benchmark, significantly enhancing model performance, generalization, and interpretability.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Large Vision Models
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Character recognition is the fundamental part of an optical character recognition (OCR) system. Word recognition, sentence transcription, document digitization, and language processing are some of the higher-order activities that can be done accurately through character recognition. Nonetheless, recognizing handwritten Bangla characters is not an easy task because they are written in different styles with inconsistent stroke patterns and a high degree of visual character resemblance. The datasets available are usually limited in intra-class and inequitable in class distribution. We have constructed a new balanced dataset of Bangla written characters to overcome those problems. This consists of 78 classes and each class has approximately 650 samples. It contains the basic characters, composite (Juktobarno) characters and numerals. The samples were a diverse group comprising a large age range and socioeconomic groups. Elementary and high school students, university students, and professionals are the contributing factors. The sample also has right and left-handed writers. We have further proposed an interaction-aware hybrid deep learning architecture that integrates EfficientNetB3, Vision Transformer, and Conformer modules in parallel. A multi-head cross-attention fusion mechanism enables effective feature interaction across these components. The proposed model achieves 98.84% accuracy on the constructed dataset and 96.49% on the external CHBCR benchmark, demonstrating strong generalization capability. Grad-CAM visualizations further provide interpretability by highlighting discriminative regions. The dataset and source code of this research is publicly available at: https://huggingface.co/MIRZARAQUIB/Bangla_Handwritten_Character_Recognition.

Problem

• Bangla Handwritten Character Recognition
• Optical Character Recognition
• Class Imbalance
• Intra-class Diversity
• Visual Similarity

Innovation

• Multi-Head Cross-Attention
• Interaction-Aware Architecture
• Bangla Handwritten Character Recognition
• Hybrid Deep Learning
• Balanced Dataset