Impact of Sunglasses on One-to-Many Facial Identification Accuracy

2024-12-07

AI Summary

This study systematically quantifies the independent and synergistic degradation effects of dark sunglasses on one-to-many face recognition accuracy, revealing performance drops comparable to severe blur or low resolution. Method: We propose a model-agnostic sunglass image synthesis augmentation technique that injects controllable sunglass artifacts into gallery images without retraining the recognition model. We further introduce the first unified degradation modeling framework jointly characterizing sunglasses with blur and downsampling, and conduct cross-degradation accuracy evaluation across mainstream face recognition models. Contributions/Results: Experiments demonstrate (1) significant accuracy degradation induced by sunglasses alone; (2) drastic performance collapse under combined degradations; (3) substantial robustness improvement—recovering ~38% accuracy loss—via synthesized sunglass-augmented galleries; and (4) severe generalization limitations arising from the scarcity of sunglass-wearing samples in training data.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

One-to-many facial identification is documented to achieve high accuracy in the case where both the probe and the gallery are"mugshot quality"images. However, an increasing number of documented instances of wrongful arrest following one-to-many facial identification have raised questions about its accuracy. Probe images used in one-to-many facial identification are often cropped from frames of surveillance video and deviate from"mugshot quality"in various ways. This paper systematically explores how the accuracy of one-to-many facial identification is degraded by the person in the probe image choosing to wear dark sunglasses. We show that sunglasses degrade accuracy for mugshot-quality images by an amount similar to strong blur or noticeably lower resolution. Further, we demonstrate that the combination of sunglasses with blur or lower resolution results in even more pronounced loss in accuracy. These results have important implications for developing objective criteria to qualify a probe image for the level of accuracy to be expected if it used for one-to-many identification. To ameliorate the accuracy degradation caused by dark sunglasses, we show that it is possible to recover about 38% of the lost accuracy by synthetically adding sunglasses to all the gallery images, without model re-training. We also show that the frequency of wearing-sunglasses images is very low in existing training sets, and that increasing the representation of wearing-sunglasses images can greatly reduce the error rate. The image set assembled for this research is available at https://cvrl.nd.edu/projects/data/ to support replication and further research.

Problem

• Sunglasses Impact
• Facial Recognition Accuracy
• Objective Image Quality Standards

Innovation

• FacialIdentificationAccuracy
• SunglassesImpact
• ProbeImageQualification