RL-TIME: Reinforcement Learning-based Task Replication in Multicore Embedded Systems

2025-03-16

AI Summary

To address the trade-offs among reliability, real-time schedulability, thermal safety, and power consumption in task replication for multicore embedded systems, this paper proposes a dynamic task replication optimization method based on Proximal Policy Optimization (PPO), a deep reinforcement learning algorithm. It is the first work to employ reinforcement learning for runtime adaptive determination of replica count, jointly optimizing system reliability objectives and core-level Thermal-Safe Power (TSP) constraints. The approach integrates real-time schedulability analysis, physics-aware thermal modeling, and power-aware task mapping into a unified framework. Compared with state-of-the-art methods, the proposed solution reduces power consumption by 63%, improves schedulability by 53%, and increases TSP constraint satisfaction rate by 72%. It effectively mitigates resource underutilization and thermal violation risks inherent in static replication strategies.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Embedded systems power many modern applications and must often meet strict reliability, real-time, thermal, and power requirements. Task replication can improve reliability by duplicating a task's execution to handle transient and permanent faults, but blindly applying replication often leads to excessive overhead and higher temperatures. Existing design-time methods typically choose the number of replicas based on worst-case conditions, which can waste resources under normal operation. In this paper, we present RL-TIME, a reinforcement learning-based approach that dynamically decides the number of replicas according to actual system conditions. By considering both the reliability target and a core-level Thermal Safe Power (TSP) constraint at run-time, RL-TIME adapts the replication strategy to avoid unnecessary overhead and overheating. Experimental results show that, compared to state-of-the-art methods, RL-TIME reduces power consumption by 63%, increases schedulability by 53%, and respects TSP 72% more often.

Problem

• Dynamic task replication in multicore embedded systems
• Optimizing reliability and thermal constraints using reinforcement learning
• Reducing power consumption and improving schedulability

Innovation

• Reinforcement learning dynamically adjusts task replication.
• Adapts replication based on real-time thermal constraints.
• Reduces power and increases system schedulability significantly.