Identity Documents Recognition and Detection using Semantic Segmentation with Convolutional Neural Network (short paper)

2025-03-03

CPITS I

AI Summary

Real-time ID document detection and recognition in complex scenes remains challenging on resource-constrained industrial single-board computers and smartphones. Method: We propose a novel lightweight CNN architecture for semantic segmentation, integrating mobile-optimized training strategies and an IoU-threshold-driven localization evaluation framework. A dedicated dataset comprising 50 document classes and 500 video clips is constructed to support robust training and benchmarking. Contribution/Results: To our knowledge, this is the first work to demonstrate feasible ID detection at high IoU (≥0.8) on edge devices, achieving detection accuracy above 0.75 while reducing model parameters and memory footprint by orders of magnitude. The architecture achieves a rigorous balance between precise localization and extreme model lightness—enabling low-power, industrial-grade deployment of document recognition on edge hardware.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Object Detection & Categorization
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Object recognition and detection are well-studied problems with a developed set of almost standard solutions. Identity documents recognition, classification, detection, and localization are the tasks required in a number of applications, particularly, in physical access control security systems at critical infrastructure premises. In this paper, we propose the new original architecture of a model based on an artificial convolutional neural network and semantic segmentation approach for the recognition and detection of identity documents in images. The challenge with the processing of such images is the limited computational performance and the limited amount of memory when such an application is running on industrial oneboard microcomputer hardware. The aim of this research is to prove the feasibility of the proposed technique and to obtain quality metrics. The methodology of the research is to evaluate the deep learning detection model trained on the mobile identity document video dataset. The dataset contains five hundred video clips for fifty different identity document types. The numerical results from simulations are used to evaluate the quality metrics. We present the results as accuracy versus threshold of the intersection over union value. The paper reports an accuracy above 0.75 for the intersection over union (IoU) threshold value of 0.8. Besides, we assessed the size of the model and proved the feasibility of running the model on an industrial one-board microcomputer or smartphone hardware.

Problem

• Develops CNN-based model for identity document recognition and detection.
• Addresses computational and memory limitations on microcomputer hardware.
• Evaluates model performance using accuracy and IoU metrics.

Innovation

• Convolutional neural network for document recognition
• Semantic segmentation for identity document detection
• Optimized for industrial microcomputer hardware performance