Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting?

2026-07-14

AI Summary

This study investigates whether high-complexity spatial attention mechanisms are indispensable for modeling global spatial dependencies in traffic forecasting. By constructing a controlled ablation framework that isolates the spatial mixing module, the authors compare uniform global aggregation against standard spatial attention across six benchmark datasets. Theoretical analysis reveals that spatial attention can be decomposed into a uniform global background component and a non-uniform residual term, with the latter offering limited performance gains that are highly dataset-dependent. Empirical results demonstrate that the uniform mixing operator incurs only a marginal 0.14% increase in average MAE compared to attention-based methods while reducing computational complexity from O(N²) to O(N). These findings challenge the necessity of Transformer-like architectures for this task and show that simple operators can effectively capture global spatial information.

Abstract

Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each individual node and all nodes across the traffic network. However, the underlying mechanism by which such global information is modeled and extracted remains insufficiently investigated. Whether global information must be extracted by high-degree-of-freedom adaptive attention or can be captured by a simple global aggregation operator remains unclear. For this purpose, we design a controlled ablation framework that replaces only the spatial mixing module to test attention-based global interaction. Across six traffic benchmarks, uniform full-range mixing and standard spatial attention each achieve lower MAE on three datasets, with only a 0.14% difference in mean MAE, while the former reduces node-scale spatial mixing complexity from O(N2) to O(N). Mechanism analysis further decomposes spatial attention into a row-uniform global background and a non-uniform residual. The residual shows dataset-dependent marginal value, suggesting that spatial attention should be justified by stable gains beyond a row-uniform global background. The corresponding source code is publicly available at: https://github.com/uuesti/U-Trans

Problem

• traffic forecasting
• global spatial information
• spatial attention
• global aggregation
• spatial dependencies

Innovation

• traffic forecasting
• spatial attention
• global aggregation
• computational complexity
• ablation study