ProgResViT: Progressive Resolution and Width for Adaptive Vision Transformers

2026-09-02

AI Summary

本文提出ProgResViT，通过逐步提高图像分辨率和模型宽度来适应性地处理图像分类问题，以实现更好的准确率与计算量之间的平衡。

Abstract

Vision Transformers (ViTs) typically process every image using a fixed input resolution and model width, even though many images can be classified with substantially less computation. We introduce ProgResViT, an input-adaptive ViT that performs inference progressively across multiple rounds. The first round processes a low-resolution image with a narrow subnetwork. Inference terminates when the prediction is sufficiently confident; otherwise, the model reuses the representations produced in the current round and proceeds with a higher-resolution input and a wider subnetwork to refine its prediction. As all rounds share a single backbone, we propose Progress-Conditioned Soft Gating (PSG), which conditions token fusion and layer outputs on the current round, block, and input resolution. On image classification, applying ProgResViT to DeiT yields better accuracy-compute trade-offs than adaptive-width, adaptive-depth, and dynamic-token baselines. With knowledge distillation, a DeiT-based ProgResViT achieves 84.9% top-1 accuracy, slightly exceeding the reported DeiT-III-S accuracy under a comparable evaluation setting. We show that the same design also provides favorable accuracy-compute trade-offs for self-supervised DINO representations and downstream semantic segmentation. Code is available at https://github.com/ds-kiel/ProgResViT.

Problem

• Vision Transformers
• input resolution
• model width
• computation

Innovation

• Progressive Resolution
• Adaptive Vision Transformers
• Progress-Conditioned Soft Gating
• Efficient Inference