Comparative Study of CNN Architectures for Binary Classification of Horses and Motorcycles in the VOC 2008 Dataset

2025-11-06

AI Summary

This study addresses the severe class imbalance in the horse vs. motorcycle binary classification task on PASCAL VOC 2008. We systematically evaluate nine state-of-the-art CNN architectures—including ConvNeXt-Tiny, ResNet-50, DenseNet-121, and ViT—under minority-class–specific data augmentation. Employing a unified training protocol and rigorous evaluation, we find ConvNeXt-Tiny achieves the highest mean average precision (mAP): 95.53% for horses and 89.12% for motorcycles. Empirical results demonstrate that targeted augmentation significantly improves minority-class detection, particularly boosting motorcycle recall. To our knowledge, this is the first work to quantitatively benchmark the robustness of diverse modern architectures to class imbalance in a fine-grained binary object classification setting. Our reproducible experimental framework establishes a practical, lightweight, and high-accuracy baseline for imbalanced visual classification, offering actionable insights for architecture selection and data-augmentation strategy design.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Object Detection & Categorization
• Search and Optimization: Learning to Search

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

This paper presents a comprehensive evaluation of nine convolutional neural network architectures for binary classification of horses and motorcycles in the VOC 2008 dataset. We address the significant class imbalance problem by implementing minority-class augmentation techniques. Our experiments compare modern architectures including ResNet-50, ConvNeXt-Tiny, DenseNet-121, and Vision Transformer across multiple performance metrics. Results demonstrate substantial performance variations, with ConvNeXt-Tiny achieving the highest Average Precision (AP) of 95.53% for horse detection and 89.12% for motorcycle detection. We observe that data augmentation significantly improves minority class detection, particularly benefiting deeper architectures. This study provides insights into architecture selection for imbalanced binary classification tasks and quantifies the impact of data augmentation strategies in mitigating class imbalance issues in object detection.

Problem

• Evaluating CNN architectures for binary classification of horses and motorcycles
• Addressing class imbalance through minority-class augmentation techniques
• Comparing performance variations across modern neural network architectures

Innovation

• Used nine CNN architectures for binary classification
• Implemented minority-class data augmentation techniques
• Compared performance using multiple evaluation metrics