RLER-TTE: An Efficient and Effective Framework for En Route Travel Time Estimation with Reinforcement Learning

2025-01-26

AI Summary

To address the low accuracy and slow inference inherent in real-time en-route travel time estimation (ER-TTE) under dynamic traffic conditions, this paper proposes a decision-prediction dual-module architecture coupled with a reinforcement learning–driven online scheduling mechanism. We introduce the first “decision-maker–predictor” collaborative framework, formulating real-time path scheduling as a Markov decision process. The framework integrates Proximal Policy Optimization (PPO) for policy learning, attention-based spatiotemporal feature encoding, dynamic model invocation strategies, and spatiotemporal curriculum learning to achieve end-to-end joint optimization. Evaluated on three real-world traffic datasets, our method reduces average prediction error by 18.7% and accelerates inference speed by 3.2× over state-of-the-art approaches, significantly enhancing both dynamic adaptability and computational efficiency.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

En Route Travel Time Estimation (ER-TTE) aims to learn driving patterns from traveled routes to achieve rapid and accurate real-time predictions. However, existing methods ignore the complexity and dynamism of real-world traffic systems, resulting in significant gaps in efficiency and accuracy in real-time scenarios. Addressing this issue is a critical yet challenging task. This paper proposes a novel framework that redefines the implementation path of ER-TTE to achieve highly efficient and effective predictions. Firstly, we introduce a novel pipeline consisting of a Decision Maker and a Predictor to rectify the inefficient prediction strategies of current methods. The Decision Maker performs efficient real-time decisions to determine whether the high-complexity prediction model in the Predictor needs to be invoked, and the Predictor recalculates the travel time or infers from historical prediction results based on these decisions. Next, to tackle the dynamic and uncertain real-time scenarios, we model the online decision-making problem as a Markov decision process and design an intelligent agent based on reinforcement learning for autonomous decision-making. Moreover, to fully exploit the spatio-temporal correlation between online data and offline data, we meticulously design feature representation and encoding techniques based on the attention mechanism. Finally, to improve the flawed training and evaluation strategies of existing methods, we propose an end-to-end training and evaluation approach, incorporating curriculum learning strategies to manage spatio-temporal data for more advanced training algorithms. Extensive evaluations on three real-world datasets confirm that our method significantly outperforms state-of-the-art solutions in both accuracy and efficiency.

Problem

• Travel Time Estimation
• Accuracy
• Efficiency

Innovation

• Reinforcement Learning
• Attention Mechanism
• Markov Decision Process