Extracting and Analyzing Rail Crossing Behavior Signatures from Videos using Tensor Methods

2026-02-17

AI Summary

This study addresses the challenge of identifying common behavioral patterns among drivers at railway crossings across diverse locations—a limitation that hinders systematic safety interventions. To this end, the authors propose a multi-view tensor decomposition framework, which, for the first time, applies non-negative symmetric CANDECOMP/PARAFAC (CP) decomposition to model multi-stage driver behavior at railway crossings. Specifically, TimeSformer is employed to extract behavioral embeddings from three critical phases: approach, waiting, and clearance. Phase-specific similarity matrices are then constructed and jointly decomposed via tensor factorization to enable automatic clustering of cross-location behavioral patterns. The analysis reveals that location exerts a significantly stronger influence on driving behavior than temporal factors and uncovers highly discriminative behavioral signatures in the approach phase. These findings establish a scalable, data-driven foundation for targeted traffic safety interventions.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Machine Learning: Matrix & Tensor Methods

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Railway crossings present complex safety challenges where driver behavior varies by location, time, and conditions. Traditional approaches analyze crossings individually, limiting the ability to identify shared behavioral patterns across locations. We propose a multi-view tensor decomposition framework that captures behavioral similarities across three temporal phases: Approach (warning activation to gate lowering), Waiting (gates down to train passage), and Clearance (train passage to gate raising). We analyze railway crossing videos from multiple locations using TimeSformer embeddings to represent each phase. By constructing phase-specific similarity matrices and applying non-negative symmetric CP decomposition, we discover latent behavioral components with distinct temporal signatures. Our tensor analysis reveals that crossing location appears to be a stronger determinant of behavior patterns than time of day, and that approach-phase behavior provides particularly discriminative signatures. Visualization of the learned component space confirms location-based clustering, with certain crossings forming distinct behavioral clusters. This automated framework enables scalable pattern discovery across multiple crossings, providing a foundation for grouping locations by behavioral similarity to inform targeted safety interventions.

Problem

• rail crossing behavior
• behavioral patterns
• video analysis
• tensor methods
• safety interventions

Innovation

• tensor decomposition
• rail crossing behavior
• TimeSformer embeddings
• behavioral signatures
• multi-view analysis