Correcting Autonomous Driving Object Detection Misclassifications with Automated Commonsense Reasoning

2026-01-07

Electronic Proceedings in Theoretical Computer Science

AI Summary

This work addresses the limitations of Level 5 autonomous driving in anomalous road scenarios—such as malfunctioning traffic lights or sudden obstacles—where perception models often fail due to misclassification stemming from scarce training data. To mitigate this, the authors propose a hybrid architecture that integrates commonsense reasoning with deep learning: when the perception model exhibits high output uncertainty, a commonsense reasoning mechanism is automatically triggered to refine object detection results. Implemented on the CARLA simulation platform, the approach combines uncertainty quantification, deep object detection, and an automated commonsense reasoning system. Experimental results demonstrate significant improvements in traffic light state recognition and obstacle detection accuracy under rare and challenging conditions, thereby validating the efficacy of commonsense augmentation in enhancing the robustness of autonomous driving perception systems.

Technology Category

• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Autonomous Vehicle (AV) technology has been heavily researched and sought after, yet there are no SAE Level 5 AVs available today in the marketplace. We contend that over-reliance on machine learning technology is the main reason. Use of automated commonsense reasoning technology, we believe, can help achieve SAE Level 5 autonomy. In this paper, we show how automated common-sense reasoning technology can be deployed in situations where there are not enough data samples available to train a deep learning-based AV model that can handle certain abnormal road scenarios. Specifically, we consider two situations where (i) a traffic signal is malfunctioning at an intersection and (ii) all the cars ahead are slowing down and steering away due to an unexpected obstruction (e.g., animals on the road). We show that in such situations, our commonsense reasoning-based solution accurately detects traffic light colors and obstacles not correctly captured by the AV's perception model. We also provide a pathway for efficiently invoking commonsense reasoning by measuring uncertainty in the computer vision model and using commonsense reasoning to handle uncertain scenarios. We describe our experiments conducted using the CARLA simulator and the results obtained. The main contribution of our research is to show that automated commonsense reasoning effectively corrects AV-based object detection misclassifications and that hybrid models provide an effective pathway to improving AV perception.

Problem

• autonomous driving
• object detection misclassification
• commonsense reasoning
• abnormal road scenarios
• perception uncertainty

Innovation

• commonsense reasoning
• autonomous driving
• object detection
• uncertainty-aware perception
• hybrid AI