DECOFFEE: Decentralized Reinforcement Learning for Time-critical Workload Offloading and Energy Efficiency across the Computing Continuum

2026-04-27

AI Summary

This study addresses the challenge of dynamically deciding whether to execute latency-sensitive, energy-constrained IoT tasks locally or offload them to neighboring edge nodes or the cloud within an edge–cloud continuum, while strictly adhering to deadline constraints to jointly minimize latency, energy consumption, and task drop rate. To this end, the authors propose a decentralized multi-agent reinforcement learning framework in which each edge node independently learns an optimal workload placement policy using a Dueling Double Deep Q-Network (Dueling DQN), informed by local observations and LSTM-predicted network states. This approach pioneers the integration of LSTM-enhanced Dueling DQN into decentralized edge computing, effectively solving the non-convex optimization problem inherent in dynamic, heterogeneous environments. Experimental results demonstrate that the proposed method consistently outperforms conventional rule-based and heuristic strategies across diverse traffic and network conditions, achieving significant reductions in latency, energy usage, and task drop rate.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The rapid proliferation of latency-sensitive and battery-constrained Internet-of-Things (IoT) applications has intensified the need for intelligent workload placement mechanisms across the Edge-Cloud computing continuum. In such environments, far-edge nodes must dynamically decide whether to execute workloads locally or offload them to neighboring nodes or the cloud, while accounting for execution delay, energy consumption, and strict timeout constraints. However, workload placement in large-scale distributed infrastructures is a highly dynamic and non-convex optimization problem due to stochastic arrivals, heterogeneous computing capacities, and time-varying network conditions. This paper proposes DECOFFEE, a decentralized reinforcement learning framework for time-critical workload offloading and energy-efficient operation across the computing continuum. The proposed multi-agent learning scheme jointly optimizes system delay, energy consumption, and workload drop rate through adaptive placement decisions. Each edge agent operates as an autonomous learning entity that derives an optimal policy from local system observations and predicted network conditions. The workload placement process is formulated as parallel Markov Decision Processes and solved using a Double Dueling Deep Q-Network (DQN) architecture enhanced with Long Short-Term Memory (LSTM) forecasting to anticipate future load conditions. Extensive simulations demonstrate that DECOFFEE and its variants consistently outperform conventional rule-based and heuristic placement strategies, achieving significant reductions in delay, energy consumption, and workload drop rate under varying traffic and network conditions.

Problem

• workload offloading
• energy efficiency
• time-critical
• edge-cloud continuum
• IoT

Innovation

• Decentralized Reinforcement Learning
• Workload Offloading
• Edge-Cloud Continuum
• Double Dueling DQN
• LSTM Forecasting