🤖 AI Summary
To address the real-time non-maximum suppression (NMS) bottleneck caused by thousands of overlapping candidate bounding boxes in embedded GPU-based object detection (e.g., NVIDIA Tegra X1/X2), this paper proposes a workload-balanced parallel greedy NMS algorithm. The method introduces a lightweight, dynamically load-partitioned CUDA kernel design—first of its kind—supporting variants such as FeatureNMS and Soft-NMS, thereby balancing accuracy and scalability. It achieves high optimization through deep architectural co-design with embedded GPU constraints, including memory bandwidth limitations and compute unit characteristics, enabling efficient thread scheduling and memory access patterns. On Tegra X1/X2, processing 1,024 candidate boxes requires only ~1 ms—14× to 40× faster than current state-of-the-art CNN-based learned NMS methods—significantly enhancing end-side real-time detection throughput and efficiency.
📝 Abstract
In the context of object detection, sliding-window classifiers and single-shot convolutional neural network (CNN) meta-architectures typically yield multiple overlapping candidate windows with similar high scores around the true location of a particular object. Non-maximum suppression (NMS) is the process of selecting a single representative candidate within this cluster of detections, so as to obtain a unique detection per object appearing on a given picture. In this paper, we present a highly scalable NMS algorithm for embedded graphics processing unit (GPU) architectures that is designed from scratch to handle workloads featuring thousands of simultaneous detections on a given picture. Our kernels are directly applicable to other sequential NMS algorithms such as FeatureNMS, Soft-NMS or AdaptiveNMS that share the inner workings of the classic greedy NMS method. The obtained performance results show that our parallel NMS algorithm is capable of clustering 1024 simultaneous detected objects per frame in roughly 1 ms on both Tegra X1 and Tegra X2 on-die GPUs, while taking 2 ms on Tegra K1. Furthermore, our proposed parallel greedy NMS algorithm yields a 14–40x speed up when compared to state-of-the-art NMS methods that require learning a CNN from annotated data.