AQ4SViT: An Automated Quantization Framework with Search Gating Policy for Compressing Spiking Vision Transformers

2026-06-13

AI Summary

This work addresses the challenge of deploying Spiking Vision Transformers (SViTs) on resource-constrained devices, where their large model size and the inefficiency of existing quantization methods—often reliant on manual hyperparameter tuning—hinder scalability. To overcome this, we propose AQ4SViT, the first automated quantization framework tailored for SViTs. AQ4SViT innovatively employs membrane potential drift as a proxy metric for accuracy and integrates a search gating mechanism with Greedy and Beam search strategies to efficiently generate quantization configurations that meet target accuracy constraints. Experiments demonstrate that AQ4SViT-Greedy achieves a 6.6× speedup in search time and reduces memory usage by 82.5%, while AQ4SViT-Beam cuts memory consumption by up to 90%, with both variants incurring no more than a 1.5% accuracy drop on ImageNet.

Abstract

Spiking Vision Transformers (SViTs) have emerged as alternative low-power ViT models, but their large sizes hinder their deployments on resource-constrained embedded AI systems. To address this, state-of-the-art works proposed quantization techniques to compress SViT models, but their manual, human-guided approach needs a huge design time and power/energy consumption to find the appropriate quantization setting for each given network, making this approach not scalable for quantizing multiple networks. Toward this, we propose AQ4SViT, a novel automated quantization framework for SViTs that can provide quick quantization settings with good trade-offs between accuracy and memory. To achieve this, AQ4SViT employs the following key ideas: quantization search strategy that evaluates the quantization setting candidates while considering the accuracy constraint; and search gating policy that quickly evaluates and selects promising quantization candidates by leveraging membrane potential drift as a performance proxy. In the search gating policy, AQSViT employs two search algorithm variants to provide trade-off options: Greedy search, which performs fast but may lead to local optima; and Beam search, which performs slower but has better performance in finding global optima selection due to a wider search space. Experimental results show that AQ4SViT-Greedy quickly finds the appropriate quantization settings, achieving up to 6.6x faster search time and up to 82.5% memory saving compared to the state-of-the-art; while AQ4SViT-Beam further reduces the memory footprint by up to 90% compared to the state-of-the-art, but with 4.5x longer search time; all these results are obtained while maintaining high accuracy within 1.5% from the original/non-quantized models on the ImageNet dataset. These results highlight that AQ4SViT framework offers advancements toward SViT deployments on embedded AI systems.

Problem

• Spiking Vision Transformers
• quantization
• embedded AI systems
• model compression
• automated quantization

Innovation

• automated quantization
• Spiking Vision Transformers
• search gating policy
• membrane potential drift
• quantization search