Semantic4Safety: Causal Insights from Zero-shot Street View Imagery Segmentation for Urban Road Safety

2025-10-17

AI Summary

This study addresses the problem of constructing interpretable, street-level safety metrics from street-view imagery (SVI) and quantifying their causal effects on distinct traffic crash types. Methodologically, we propose Semantic4Safety—a novel framework integrating zero-shot semantic segmentation (to extract 11 fine-grained street features), an XGBoost multi-class classifier, and SHAP-based interpretability analysis, coupled with generalized propensity score weighting to estimate average treatment effects (ATE) for crash-type-specific causal inference across ~30,000 real-world crashes. Results reveal that scene complexity, exposure level, and road geometric features are the most predictive; larger drivable and emergency buffer areas significantly reduce crash risk, whereas excessive visual openness paradoxically increases it. Our contribution lies in establishing a causally grounded, attribution-aware, and generalizable visual analytics paradigm for urban traffic safety diagnosis and targeted intervention.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Natural Language Processing: Safety and Robustness
• Reasoning under Uncertainty: Causality

Application Category

• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Street-view imagery (SVI) offers a fine-grained lens on traffic risk, yet two fundamental challenges persist: (1) how to construct street-level indicators that capture accident-related features, and (2) how to quantify their causal impacts across different accident types. To address these challenges, we propose Semantic4Safety, a framework that applies zero-shot semantic segmentation to SVIs to derive 11 interpretable streetscape indicators, and integrates road type as contextual information to analyze approximately 30,000 accident records in Austin. Specifically, we train an eXtreme Gradient Boosting (XGBoost) multi-class classifier and use Shapley Additive Explanations (SHAP) to interpret both global and local feature contributions, and then apply Generalized Propensity Score (GPS) weighting and Average Treatment Effect (ATE) estimation to control confounding and quantify causal effects. Results uncover heterogeneous, accident-type-specific causal patterns: features capturing scene complexity, exposure, and roadway geometry dominate predictive power; larger drivable area and emergency space reduce risk, whereas excessive visual openness can increase it. By bridging predictive modeling with causal inference, Semantic4Safety supports targeted interventions and high-risk corridor diagnosis, offering a scalable, data-informed tool for urban road safety planning.

Problem

• Construct street-level indicators capturing accident-related features from imagery
• Quantify causal impacts of streetscape indicators across accident types
• Bridge predictive modeling with causal inference for road safety planning

Innovation

• Zero-shot semantic segmentation for streetscape indicators
• XGBoost and SHAP for feature interpretation
• GPS weighting and ATE for causal effect estimation