Problem
Research questions and friction points this paper is trying to address.
quantization-aware training
edge devices
modality imbalance
knowledge distillation
cross-modal alignment
Efficient Quantization-Aware Distillation with Cross-Modal Alignment for Edge Vision-Language Models
🤖 AI Summary
本文提出一种统一框架，通过联合优化知识蒸馏和量化来解决边缘设备上视觉-语言模型的部署问题，并设计了跨注意力适配器增强非RGB模态。
📝 Abstract
Large-scale vision-language models (VLM) such as CLIP enable strong open-vocabulary reasoning, yet deploying these capabilities on resource-constrained edge devices remains challenging. EdgeVL addresses this problem by distilling CLIP representations into lightweight multi-modal encoders and applying quantization-aware training (QAT) for efficient Open-Vocabulary Classification (OVC) on edge hardware. However, its two-stage optimization applies different objectives for distillation and QAT, and contrastive learning is performed within the quantized student space, which can result in inconsistent optimization and reduced training efficiency. Moreover, identical supervision across RGB and non-RGB modalities may lead to modality imbalance. We propose a unified framework for quantized semantic distillation tailored to edge deployment. By jointly optimizing distillation and quantization within a unified teacher-anchored framework, our method ensures consistent training under quantization, suppressing hard negatives and enlarging decision margins. Additionally, we design a lightweight cross-attention adapter that enhances non-RGB representations through RGB-guided semantic transfer, narrowing the modality gap. Extensive experiments demonstrate consistent improvements on non-RGB modalities while maintaining deployment efficiency.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Quantization-Aware Distillation
Cross-Modal Alignment
Edge Deployment
Unified Framework
Lightweight Cross-Attention Adapter
🔎 Similar Papers
No similar papers found.
