AIVD: Adaptive Edge-Cloud Collaboration for Accurate and Efficient Industrial Visual Detection

2026-01-08

arXiv.org

AI Summary

This work addresses the challenges of insufficient object localization accuracy and inefficient edge-cloud collaborative deployment in industrial visual inspection using multimodal large language models (MLLMs). The authors propose a novel edge-cloud协同 architecture wherein a lightweight edge detector performs high-precision object localization, while a cloud-based MLLM generates high-quality semantic descriptions. To enhance model robustness, they design a vision-language co-enhanced fine-tuning strategy and introduce a heterogeneous resource-aware dynamic scheduling algorithm that adaptively responds to variations in device capabilities and network conditions. Experimental results demonstrate that the proposed approach significantly reduces resource consumption while simultaneously improving classification accuracy, semantic generation quality, system throughput, and end-to-end latency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Multimodal large language models (MLLMs) demonstrate exceptional capabilities in semantic understanding and visual reasoning, yet they still face challenges in precise object localization and resource-constrained edge-cloud deployment. To address this, this paper proposes the AIVD framework, which achieves unified precise localization and high-quality semantic generation through the collaboration between lightweight edge detectors and cloud-based MLLMs. To enhance the cloud MLLM's robustness against edge cropped-box noise and scenario variations, we design an efficient fine-tuning strategy with visual-semantic collaborative augmentation, significantly improving classification accuracy and semantic consistency. Furthermore, to maintain high throughput and low latency across heterogeneous edge devices and dynamic network conditions, we propose a heterogeneous resource-aware dynamic scheduling algorithm. Experimental results demonstrate that AIVD substantially reduces resource consumption while improving MLLM classification performance and semantic generation quality. The proposed scheduling strategy also achieves higher throughput and lower latency across diverse scenarios.

Problem

• multimodal large language models
• object localization
• edge-cloud collaboration
• resource-constrained deployment
• industrial visual detection

Innovation

• edge-cloud collaboration
• multimodal large language models
• visual-semantic augmentation
• dynamic scheduling
• industrial visual detection