YOLO Evolution: A Comprehensive Benchmark and Architectural Review of YOLOv12, YOLO11, and Their Previous Versions

2024-10-31

AI Summary

Detecting small, occluded, and multi-scale objects—such as traffic signs, African wildlife, and small vessels—remains challenging for real-time object detectors. Method: This work systematically benchmarks YOLOv3 through YOLOv11 across these three demanding datasets under a unified evaluation framework, measuring detection accuracy (mAP₅₀₋₉₅), latency, computational cost (GFLOPs), and model size (MB). Contribution/Results: It presents the first comprehensive empirical analysis of the YOLOv11 series, identifying YOLOv11m as the Pareto-optimal variant: achieving mAP₅₀₋₉₅ of 0.795, 0.810, and 0.325 on the respective datasets; attaining 2.4 ms average inference latency; requiring only 38.8 MB storage and 67.6 GFLOPs. Compared to YOLOv9 (accurate but inefficient) and YOLOv10 (fast but weak on occlusion), YOLOv11m establishes a new accuracy-efficiency trade-off benchmark. These findings provide evidence-based guidance for detector selection, lightweight design, and small-object detection optimization.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Hardware-aware ML
• Search and Optimization: Evaluation and Analysis

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

This study presents a comprehensive benchmark analysis of various YOLO (You Only Look Once) algorithms, from YOLOv3 to the newest addition. It represents the first research to comprehensively evaluate the performance of YOLO11, the latest addition to the YOLO family. It evaluates their performance on three diverse datasets: Traffic Signs (with varying object sizes), African Wildlife (with diverse aspect ratios and at least one instance of the object per image), and Ships and Vessels (with small-sized objects of a single class), ensuring a comprehensive assessment across datasets with distinct challenges. To ensure a robust evaluation, we employ a comprehensive set of metrics, including Precision, Recall, Mean Average Precision (mAP), Processing Time, GFLOPs count, and Model Size. Our analysis highlights the distinctive strengths and limitations of each YOLO version. For example: YOLOv9 demonstrates substantial accuracy but struggles with detecting small objects and efficiency whereas YOLOv10 exhibits relatively lower accuracy due to architectural choices that affect its performance in overlapping object detection but excels in speed and efficiency. Additionally, the YOLO11 family consistently shows superior performance in terms of accuracy, speed, computational efficiency, and model size. YOLO11m achieved a remarkable balance of accuracy and efficiency, scoring mAP50-95 scores of 0.795, 0.81, and 0.325 on the Traffic Signs, African Wildlife, and Ships datasets, respectively, while maintaining an average inference time of 2.4ms, a model size of 38.8Mb, and around 67.6 GFLOPs on average. These results provide critical insights for both industry and academia, facilitating the selection of the most suitable YOLO algorithm for diverse applications and guiding future enhancements.

Problem

• Benchmarking YOLO algorithms from YOLOv3 to YOLOv12.
• Evaluating YOLO11 performance on diverse datasets.
• Assessing YOLO variants using multiple performance metrics.

Innovation

• Comprehensive benchmark analysis
• Diverse datasets evaluation
• Multiple performance metrics