TopoCast: A Topological Fidelity Framework for Evaluating Transformer-Based Time Series Forecasting

2026-06-24

AI Summary

This study addresses the limitations of conventional point-wise error metrics—such as mean squared error—in evaluating time series forecasts, as they often fail to capture essential dynamical characteristics like structural patterns, oscillatory behavior, and phase alignment. To overcome this, the authors propose TopoCast, a framework that constructs phase-space representations via Takens’ delay embedding and employs persistent homology to compare the topological structures of predicted and ground-truth sequences. The framework introduces a Topological Fidelity Score (TFS) and its localized variant (LTFS), the latter incorporating dominant-period overlap to enable phase-aware temporal localization in evaluation. Experiments across three real-world datasets on five Transformer-based models demonstrate that TopoCast effectively uncovers structural distortions invisible to standard metrics, thereby validating the necessity and efficacy of topology-aware assessment in time series forecasting.

Abstract

Deep learning-based models have achieved state-of-the-art performance in Time Series Forecasting (TSF), yet their evaluation remains dominated by pointwise error metrics such as Mean Squared Error (MSE), which quantify numerical accuracy but overlook structural properties of the forecast signal, including recurrent dynamics, oscillatory behavior, and phase alignment. As a result, forecasts exhibiting over-smoothing, phase shifts, or frequency distortions may achieve favorable error scores despite substantial structural degradation. To address this limitation, we propose TopoCast, a topology-driven framework for evaluating structural fidelity in TSF. TopoCast reconstructs phase-space representations of forecast and ground-truth sequences using Takens delay embedding and applies persistent homology to characterize their intrinsic dynamics. We derive four complementary topological fidelity measures from persistence diagrams and aggregate them into a Topological Fidelity Score (TFS). We further introduce dominant cycle overlap, a novel metric that maps persistent topological features to the temporal domain to assess whether dominant oscillatory patterns occur at the correct time points. Combined with TFS, this yields the Localized Topological Fidelity Score (LTFS), a phase-aware measure that captures temporal localization errors invisible to existing evaluation metrics. Experiments on five Transformer architectures across three real-world benchmark datasets demonstrate that models with similar forecasting errors can exhibit markedly different structural fidelity profiles, revealing failure modes overlooked by conventional evaluation and highlighting the value of topology-aware forecast assessment.

Problem

• time series forecasting
• evaluation metrics
• structural fidelity
• topological analysis
• phase alignment

Innovation

• Topological Fidelity
• Persistent Homology
• Time Series Forecasting
• Phase-Space Reconstruction
• Transformer Evaluation