Analyzing sequential activity and travel decisions with interpretable deep inverse reinforcement learning

2025-03-17

AI Summary

Existing research on activity-travel sequence modeling prioritizes prediction accuracy in discrete inverse reinforcement learning (DIRL) while neglecting interpretability of the underlying decision-making mechanisms. Method: We propose the first DIRL framework integrating adversarial inverse reinforcement learning (AIRL) with an interpretable surrogate model, jointly learning and decomposing both the policy function (characterizing choice probability distributions) and the reward function (disentangling short- and long-term utilities). Grounded in utility theory, our data-driven approach leverages real-world travel survey data to identify key behavioral determinants, population heterogeneity, and individual-level utility variations across activity sequences. Contribution/Results: The framework achieves dual interpretability—clarifying both *what* choices are made and *why*—significantly enhancing behavioral insight and model credibility. It bridges the gap between machine learning and behavioral theory, offering a principled, transparent foundation for human mobility modeling.

Technology Category

• Planning, Routing, and Scheduling: Model-Based Reasoning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Travel demand modeling has shifted from aggregated trip-based models to behavior-oriented activity-based models because daily trips are essentially driven by human activities. To analyze the sequential activity-travel decisions, deep inverse reinforcement learning (DIRL) has proven effective in learning the decision mechanisms by approximating a reward function to represent preferences and a policy function to replicate observed behavior using deep neural networks (DNNs). However, most existing research has focused on using DIRL to enhance only prediction accuracy, with limited exploration into interpreting the underlying decision mechanisms guiding sequential decision-making. To address this gap, we introduce an interpretable DIRL framework for analyzing activity-travel decision processes, bridging the gap between data-driven machine learning and theory-driven behavioral models. Our proposed framework adapts an adversarial IRL approach to infer the reward and policy functions of activity-travel behavior. The policy function is interpreted through a surrogate interpretable model based on choice probabilities from the policy function, while the reward function is interpreted by deriving both short-term rewards and long-term returns for various activity-travel patterns. Our analysis of real-world travel survey data reveals promising results in two key areas: (i) behavioral pattern insights from the policy function, highlighting critical factors in decision-making and variations among socio-demographic groups, and (ii) behavioral preference insights from the reward function, indicating the utility individuals gain from specific activity sequences.

Problem

• Interpret sequential activity-travel decision mechanisms using deep learning.
• Bridge gap between data-driven models and behavioral theory in travel demand.
• Infer and interpret reward and policy functions for activity-travel behavior.

Innovation

• Interpretable deep inverse reinforcement learning framework
• Adversarial IRL for reward and policy functions
• Surrogate model interprets policy and reward functions