Measurement-Driven Sub-Network Selection for On-Premise Retrieval-Augmented Factory Agents

2026-09-02

AI Summary

研究解决了工厂助手模型在硬件限制下的部署问题，通过结构压缩、检索增强适应及子网络选择方法优化模型大小、速度和质量。

Abstract

On-premise assistants can give factory workers conversational access to machine documentation, but models capable of the task rarely fit shop-floor hardware. We show that after structural compression and retrieval-grounded adaptation, model size is no longer a reliable predictor of adapted answer quality: general capability falls almost linearly with parameter count, while judged retrieval-augmented answer quality does not. We therefore treat deployment as a post-adaptation selection problem, committing one sub-network per device on judged answer quality and measured on-device throughput under a configurable general-capability floor and memory budget; rules that optimize size, speed, or quality alone each give up capability or throughput. A weight-shared supernetwork trained with sandwich-style in-place distillation keeps this selection inexpensive. In a manufacturing-manual case study, extraction costs 13.7 percent of the unpruned model's judged quality and retrieval-grounded distillation returns it to within 4.6 percent, recovering two thirds of the loss, and the same assistant runs across three heterogeneous edge tiers at 1.3 to 5 watts standby.

Problem

• on-premise assistants
• factory workers
• model size
• retrieval-augmented
• deployment

Innovation

• sub-network selection
• retrieval-grounded adaptation
• sandwich-style in-place distillation
• on-device throughput