LoRA-based Parameter-Efficient LLMs for Continuous Learning in Edge-based Malware Detection

2026-02-12

AI Summary

This work addresses the challenges of deploying large language models on resource-constrained edge devices and the limited generalization of existing approaches against continuously evolving malware threats, which often lack cross-device collaborative learning mechanisms. To overcome these limitations, we propose a LoRA-based continual learning framework that employs lightweight Transformers (e.g., DistilBERT, TinyT5) for local incremental fine-tuning at edge nodes. A federated coordinator aggregates and redistributes LoRA adapters across devices, enabling knowledge sharing without transmitting raw data. To our knowledge, this is the first integration of parameter-efficient LoRA into continual learning for edge-based malware detection. Evaluated on Edge-IIoTset and TON-IoT datasets, our method improves unknown attack detection accuracy by 20–25% over isolated fine-tuning, with only a 0.6–1.8 MB increase in model size, while maintaining stable F1 scores and loss.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Humans and AI: Human-in-the-loop Machine Learning
• Natural Language Processing: (Large) Language Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The proliferation of edge devices has created an urgent need for security solutions capable of detecting malware in real time while operating under strict computational and memory constraints. Recently, Large Language Models (LLMs) have demonstrated remarkable capabilities in recognizing complex patterns, yet their deployment on edge devices remains impractical due to their resource demands. However, in edge malware detection, static or centrally retrained models degrade under evolving threats and heterogeneous traffic; locally trained models become siloed and fail to transfer across domains. To overcome these limitations, in this paper, we present a continuous learning architecture for edge-based malware detection that combines local adaptation on each device with global knowledge sharing through parameter-efficient LoRA adapters. Lightweight transformer models (DistilBERT, DistilGPT-2, TinyT5) run on edge nodes and are incrementally fine-tuned on device-specific traffic; only the resulting LoRA modules are aggregated by a lightweight coordinator and redistributed, enabling cross-device generalization without exchanging raw data. We evaluate on two public IoT security datasets, Edge-IIoTset and TON-IoT, under multi-round learning to simulate evolving threats. Compared to isolated fine-tuning, the LoRA-based exchange yields up to 20-25% accuracy gains when models encounter previously unseen attacks from another domain, while maintaining stable loss and F1 across rounds. LoRA adds less than 1% to model size (~0.6-1.8 MB), making updates practical for constrained edge hardware.

Problem

• edge-based malware detection
• continuous learning
• parameter-efficient LLMs
• cross-device generalization
• resource-constrained devices

Innovation

• LoRA
• parameter-efficient fine-tuning
• continuous learning
• edge malware detection
• federated knowledge sharing