Offline Reinforcement Learning for Mobility Robustness Optimization

2025-06-28

AI Summary

This work addresses the low efficiency and poor adaptability of manual Cell Individual Offset (CIO) tuning in Mobility Robustness Optimization (MRO). For the first time, it introduces offline reinforcement learning (RL) to MRO optimization, proposing a dual-paradigm framework that integrates Decision Transformer (DT) with Conservative Q-Learning (CQL). Leveraging historical handover logs—including handover failures and ping-pong events—the framework performs sequence-based policy learning without online exploration. It supports flexible multi-objective modeling, significantly enhancing automation and scenario adaptability of CIO adjustment. Evaluated on a real 3.5 GHz NR network, the method reduces handover anomaly rates by up to 7%, outperforming conventional rule-based approaches in both performance and robustness. These results demonstrate the effectiveness and practical viability of offline RL for high-complexity wireless network optimization tasks.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In this work we revisit the Mobility Robustness Optimisation (MRO) algorithm and study the possibility of learning the optimal Cell Individual Offset tuning using offline Reinforcement Learning. Such methods make use of collected offline datasets to learn the optimal policy, without further exploration. We adapt and apply a sequence-based method called Decision Transformers as well as a value-based method called Conservative Q-Learning to learn the optimal policy for the same target reward as the vanilla rule-based MRO. The same input features related to failures, ping-pongs, and other handover issues are used. Evaluation for realistic New Radio networks with 3500 MHz carrier frequency on a traffic mix including diverse user service types and a specific tunable cell-pair shows that offline-RL methods outperform rule-based MRO, offering up to 7% improvement. Furthermore, offline-RL can be trained for diverse objective functions using the same available dataset, thus offering operational flexibility compared to rule-based methods.

Problem

• Optimizing Mobility Robustness using offline Reinforcement Learning
• Learning optimal Cell Individual Offset tuning without exploration
• Improving handover performance in New Radio networks

Innovation

• Offline Reinforcement Learning for MRO optimization
• Decision Transformers and Conservative Q-Learning applied
• Outperforms rule-based MRO by 7% improvement