You Only Look Once at Anytime (AnytimeYOLO): Analysis and Optimization of Early-Exits for Object-Detection

2025-03-21

AI Summary

This paper addresses the need for interruptible inference in safety-critical real-time object detection. We propose AnytimeYOLO—the first YOLO architecture explicitly designed for anytime inference. Our method introduces three key innovations: (1) a novel transposed YOLO backbone that enhances semantic expressiveness in early layers; (2) a multi-granularity early-exit mechanism coupled with a joint optimization algorithm for exit order and subset selection, enabling fine-grained, controllable early termination; and (3) a new anytime quality metric and systematic analysis of deployment cost bottlenecks. Experiments demonstrate that AnytimeYOLO significantly improves the latency–accuracy trade-off under resource constraints, while guaranteeing valid outputs at arbitrary inference timepoints. The results validate the feasibility and practicality of anytime object detection in real-world safety-critical applications.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We introduce AnytimeYOLO, a family of variants of the YOLO architecture that enables anytime object detection. Our AnytimeYOLO networks allow for interruptible inference, i.e., they provide a prediction at any point in time, a property desirable for safety-critical real-time applications. We present structured explorations to modify the YOLO architecture, enabling early termination to obtain intermediate results. We focus on providing fine-grained control through high granularity of available termination points. First, we formalize Anytime Models as a special class of prediction models that offer anytime predictions. Then, we discuss a novel transposed variant of the YOLO architecture, that changes the architecture to enable better early predictions and greater freedom for the order of processing stages. Finally, we propose two optimization algorithms that, given an anytime model, can be used to determine the optimal exit execution order and the optimal subset of early-exits to select for deployment in low-resource environments. We evaluate the anytime performance and trade-offs of design choices, proposing a new anytime quality metric for this purpose. In particular, we also discuss key challenges for anytime inference that currently make its deployment costly.

Problem

• Enabling interruptible inference for real-time object detection
• Optimizing early-exit strategies in YOLO architectures
• Balancing trade-offs in anytime model performance

Innovation

• Early-exit YOLO for interruptible inference
• Transposed YOLO for better early predictions
• Optimization algorithms for exit execution order