Learning Temporal Saliency for Time Series Forecasting with Cross-Scale Attention

2025-09-26

AI Summary

In time series forecasting, existing interpretability methods face two key bottlenecks: post-hoc significance analysis incurs high computational overhead, while intrinsically interpretable models often sacrifice predictive accuracy. To address this, we propose CrossScaleNet—a fully end-to-end trainable multi-scale forecasting model. Its core innovation is a patch-based cross-scale cross-attention mechanism that jointly optimizes temporal significance detection and forecasting, enabling intrinsic, efficient, and post-processing-free interpretation. Evaluated on synthetic data and multiple real-world benchmarks (e.g., ETT, Weather, Traffic), CrossScaleNet achieves state-of-the-art forecasting performance—significantly outperforming mainstream Transformer variants—while accurately localizing critical time segments. This demonstrates a principled unification of high predictive accuracy and strong, faithful interpretability.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Interpretability, Explainability, and Transparency
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Explainability in time series forecasting is essential for improving model transparency and supporting informed decision-making. In this work, we present CrossScaleNet, an innovative architecture that combines a patch-based cross-attention mechanism with multi-scale processing to achieve both high performance and enhanced temporal explainability. By embedding attention mechanisms into the training process, our model provides intrinsic explainability for temporal saliency, making its decision-making process more transparent. Traditional post-hoc methods for temporal saliency detection are computationally expensive, particularly when compared to feature importance detection. While ablation techniques may suffice for datasets with fewer features, identifying temporal saliency poses greater challenges due to its complexity. We validate CrossScaleNet on synthetic datasets with known saliency ground truth and on established public benchmarks, demonstrating the robustness of our method in identifying temporal saliency. Experiments on real-world datasets for forecasting task show that our approach consistently outperforms most transformer-based models, offering better explainability without sacrificing predictive accuracy. Our evaluations demonstrate superior performance in both temporal saliency detection and forecasting accuracy. Moreover, we highlight that existing models claiming explainability often fail to maintain strong performance on standard benchmarks. CrossScaleNet addresses this gap, offering a balanced approach that captures temporal saliency effectively while delivering state-of-the-art forecasting performance across datasets of varying complexity.

Problem

• Enhancing temporal explainability in time series forecasting models
• Overcoming computational limitations of traditional saliency detection methods
• Balancing high forecasting accuracy with transparent decision-making processes

Innovation

• Cross-attention mechanism with multi-scale processing
• Intrinsic explainability for temporal saliency detection
• Balanced approach maintaining state-of-the-art forecasting performance