Interpretable Deep Reinforcement Learning for Element-level Bridge Life-cycle Optimization

2026-04-02

AI Summary

This study addresses the challenge of rapidly expanding state spaces in component-level, condition-based life-cycle management of bridges, where four-dimensional condition-state distributions hinder the derivation of interpretable and optimal maintenance policies. To overcome this, the authors propose an interpretable deep reinforcement learning approach that, for the first time, employs a differentiable soft decision tree as the policy network. By integrating temperature annealing, regularization, and pruning mechanisms, the method generates deterministic, skewed decision trees with compact structures and logically coherent nodes. Evaluated on a steel-girder bridge case study, the approach achieves near-optimal performance while substantially enhancing policy interpretability and auditability, thereby demonstrating its effectiveness and practical applicability in both supervised and reinforcement learning settings.

Technology Category

• Machine Learning: Reinforcement Learning
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

The new Specifications for the National Bridge Inventory (SNBI), in effect from 2022, emphasize the use of element-level condition states (CS) for risk-based bridge management. Instead of a general component rating, element-level condition data use an array of relative CS quantities (i.e., CS proportions) to represent the condition of a bridge. Although this greatly increases the granularity of bridge condition data, it introduces challenges to set up optimal life-cycle policies due to the expanded state space from one single categorical integer to four-dimensional probability arrays. This study proposes a new interpretable reinforcement learning (RL) approach to seek optimal life-cycle policies based on element-level state representations. Compared to existing RL methods, the proposed algorithm yields life-cycle policies in the form of oblique decision trees with reasonable amounts of nodes and depth, making them directly understandable and auditable by humans and easily implementable into current bridge management systems. To achieve near-optimal policies, the proposed approach introduces three major improvements to existing RL methods: (a) the use of differentiable soft tree models as actor function approximators, (b) a temperature annealing process during training, and (c) regularization paired with pruning rules to limit policy complexity. Collectively, these improvements can yield interpretable life-cycle policies in the form of deterministic oblique decision trees. The benefits and trade-offs from these techniques are demonstrated in both supervised and reinforcement learning settings. The resulting framework is illustrated in a life-cycle optimization problem for steel girder bridges.

Problem

• element-level condition states
• bridge life-cycle optimization
• state space expansion
• risk-based bridge management
• interpretable policies

Innovation

• interpretable reinforcement learning
• oblique decision trees
• element-level bridge management
• differentiable soft trees
• policy regularization