Training A Neural Network For Partially Occluded Road Sign Identification In The Context Of Autonomous Vehicles

2025-03-23

AI Summary

This paper addresses the significant degradation in traffic sign recognition performance under partial occlusion (e.g., tree shadows, billboards) in autonomous driving. To tackle this challenge, we propose a systematic solution comprising three key components. First, we construct the first publicly available dataset—comprising 5,746 images—captured from full-field-of-view perspectives and featuring realistic partial occlusions. Second, we quantitatively validate, for the first time, the detrimental impact of occlusion on model robustness, demonstrating that training solely on unoccluded samples leads to severe generalization failure. Third, we design a lightweight CNN architecture and employ full-layer unfreezing of VGG16 with transfer learning, augmented by realistic occlusion simulation and comprehensive data augmentation. Experimental results show that our VGG16-based approach achieves 99% recognition accuracy—substantially outperforming baseline methods—and underscore the critical role of incorporating real-world occlusion data in enhancing the safety and reliability of autonomous driving systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The increasing number of autonomous vehicles and the rapid development of computer vision technologies underscore the particular importance of conducting research on the accuracy of traffic sign recognition. Numerous studies in this field have already achieved significant results, demonstrating high effectiveness in addressing traffic sign recognition tasks. However, the task becomes considerably more complex when a sign is partially obscured by surrounding objects, such as tree branches, billboards, or other elements of the urban environment. In our study, we investigated how partial occlusion of traffic signs affects their recognition. For this purpose, we collected a dataset comprising 5,746 images, including both fully visible and partially occluded signs, and made it publicly available. Using this dataset, we compared the performance of our custom convolutional neural network (CNN), which achieved 96% accuracy, with models trained using transfer learning. The best result was obtained by VGG16 with full layer unfreezing, reaching 99% accuracy. Additional experiments revealed that models trained solely on fully visible signs lose effectiveness when recognizing occluded signs. This highlights the critical importance of incorporating real-world data with partial occlusion into training sets to ensure robust model performance in complex practical scenarios and to enhance the safety of autonomous driving.

Problem

• Improving recognition of partially occluded road signs for autonomous vehicles
• Comparing custom CNN and transfer learning models for occlusion handling
• Demonstrating the necessity of occlusion-inclusive training datasets

Innovation

• Custom CNN for occluded sign recognition
• Public dataset with occluded and clear signs
• Transfer learning with VGG16 for high accuracy