MF-LPR$^2$: Multi-Frame License Plate Image Restoration and Recognition using Optical Flow

2025-08-19

AI Summary

To address license plate recognition challenges in dashcam imagery—caused by low resolution, motion blur, and glare—this paper proposes a multi-frame restoration and recognition framework that requires no pre-trained priors. Methodologically, it employs high-accuracy optical flow estimation for inter-frame alignment and introduces a novel spatiotemporal consistency detection module coupled with erroneous-flow correction to suppress artifacts and preserve evidence integrity. Subsequently, multi-frame information is fused to jointly enhance license plate appearance and enable end-to-end recognition. Evaluated on the self-collected RLPR dataset, the method achieves state-of-the-art performance: PSNR, SSIM, and LPIPS surpass those of eight existing SOTA models; recognition accuracy reaches 86.44%, outperforming the best single-frame and multi-frame baselines by +14.04% and +3.89%, respectively. These results validate the effectiveness of temporal modeling and prior-free reconstruction for robust license plate recognition.

Technology Category

• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

License plate recognition (LPR) is important for traffic law enforcement, crime investigation, and surveillance. However, license plate areas in dash cam images often suffer from low resolution, motion blur, and glare, which make accurate recognition challenging. Existing generative models that rely on pretrained priors cannot reliably restore such poor-quality images, frequently introducing severe artifacts and distortions. To address this issue, we propose a novel multi-frame license plate restoration and recognition framework, MF-LPR$^2$, which addresses ambiguities in poor-quality images by aligning and aggregating neighboring frames instead of relying on pretrained knowledge. To achieve accurate frame alignment, we employ a state-of-the-art optical flow estimator in conjunction with carefully designed algorithms that detect and correct erroneous optical flow estimations by leveraging the spatio-temporal consistency inherent in license plate image sequences. Our approach enhances both image quality and recognition accuracy while preserving the evidential content of the input images. In addition, we constructed a novel Realistic LPR (RLPR) dataset to evaluate MF-LPR$^2$. The RLPR dataset contains 200 pairs of low-quality license plate image sequences and high-quality pseudo ground-truth images, reflecting the complexities of real-world scenarios. In experiments, MF-LPR$^2$ outperformed eight recent restoration models in terms of PSNR, SSIM, and LPIPS by significant margins. In recognition, MF-LPR$^2$ achieved an accuracy of 86.44%, outperforming both the best single-frame LPR (14.04%) and the multi-frame LPR (82.55%) among the eleven baseline models. The results of ablation studies confirm that our filtering and refinement algorithms significantly contribute to these improvements.

Problem

• Restoring low-resolution, blurry, and glaring license plate images
• Addressing artifacts from pretrained generative models in restoration
• Improving recognition accuracy by multi-frame alignment and aggregation

Innovation

• Multi-frame alignment using optical flow
• Spatio-temporal consistency for error correction
• Novel realistic dataset for evaluation