Tackling Device Data Distribution Real-time Shift via Prototype-based Parameter Editing

2025-09-08

AI Summary

Severe real-time data distribution shift at the edge critically degrades the generalization of lightweight models, while existing fine-tuning methods incur prohibitive computational overhead and hinder on-device deployment. This paper proposes Persona: a prototype-based, backpropagation-free parameter editing framework that pioneers the integration of prototype clustering with parameter editing matrices. Leveraging a cloud-based neural adapter, Persona generates cross-layer editing matrices deployed directly on-device, enabling dynamic model evolution without retraining. Its core innovation lies in decoupling representation learning from parameter updates, thereby supporting real-time, data-driven adaptive inference. Extensive evaluation across diverse vision and recommendation tasks demonstrates that Persona significantly improves accuracy under distribution shift (average +3.2%) and inference efficiency (47% latency reduction), while maintaining lightweight design and broad task applicability.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

The on-device real-time data distribution shift on devices challenges the generalization of lightweight on-device models. This critical issue is often overlooked in current research, which predominantly relies on data-intensive and computationally expensive fine-tuning approaches. To tackle this, we introduce Persona, a novel personalized method using a prototype-based, backpropagation-free parameter editing framework to enhance model generalization without post-deployment retraining. Persona employs a neural adapter in the cloud to generate a parameter editing matrix based on real-time device data. This matrix adeptly adapts on-device models to the prevailing data distributions, efficiently clustering them into prototype models. The prototypes are dynamically refined via the parameter editing matrix, facilitating efficient evolution. Furthermore, the integration of cross-layer knowledge transfer ensures consistent and context-aware multi-layer parameter changes and prototype assignment. Extensive experiments on vision task and recommendation task on multiple datasets confirm Persona's effectiveness and generality.

Problem

• Addressing real-time data distribution shifts on devices
• Enhancing generalization of lightweight on-device models
• Eliminating need for post-deployment retraining requirements

Innovation

• Prototype-based parameter editing framework
• Backpropagation-free neural adapter adaptation
• Cross-layer knowledge transfer integration