Dynamic and Distributed Routing in IoT Networks based on Multi-Objective Q-Learning

2025-05-01

AI Summary

To address the multi-objective routing optimization challenge in IoT networks arising from dynamic priority switching—such as conflicts between latency sensitivity and energy efficiency—this paper proposes a distributed multi-objective Q-learning routing method. The approach introduces a novel greedy interpolation strategy that jointly approximates the Pareto front and models online preference evolution, enabling rapid and robust adaptation to abrupt preference shifts. Fully decentralized, it operates without global state information and supports real-time, adaptive decision-making. Experimental evaluation under diverse exploration strategies and sudden preference change patterns demonstrates that the proposed method significantly outperforms state-of-the-art algorithms: it achieves higher composite reward, improves energy efficiency by 18.7%, and increases packet delivery ratio by 12.3%.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The last few decades have witnessed a rapid increase in IoT devices owing to their wide range of applications, such as smart healthcare monitoring systems, smart cities, and environmental monitoring. A critical task in IoT networks is sensing and transmitting information over the network. The IoT nodes gather data by sensing the environment and then transmit this data to a destination node via multi-hop communication, following some routing protocols. These protocols are usually designed to optimize possibly contradictory objectives, such as maximizing packet delivery ratio and energy efficiency. While most literature has focused on optimizing a static objective that remains unchanged, many real-world IoT applications require adapting to rapidly shifting priorities. For example, in monitoring systems, some transmissions are time-critical and require a high priority on low latency, while other transmissions are less urgent and instead prioritize energy efficiency. To meet such dynamic demands, we propose novel dynamic and distributed routing based on multiobjective Q-learning that can adapt to changes in preferences in real-time. Our algorithm builds on ideas from both multi-objective optimization and Q-learning. We also propose a novel greedy interpolation policy scheme to take near-optimal decisions for unexpected preference changes. The proposed scheme can approximate and utilize the Pareto-efficient solutions for dynamic preferences, thus utilizing past knowledge to adapt to unpredictable preferences quickly during runtime. Simulation results show that the proposed scheme outperforms state-of-the-art algorithms for various exploration strategies, preference variation patterns, and important metrics like overall reward, energy efficiency, and packet delivery ratio.

Problem

• Optimizing dynamic routing in IoT networks with conflicting objectives
• Adapting to real-time priority shifts in IoT data transmission
• Balancing packet delivery ratio and energy efficiency dynamically

Innovation

• Multi-objective Q-learning for dynamic routing
• Greedy interpolation policy for preference changes
• Pareto-efficient solutions for adaptive preferences