Sortability of Time Series Data

2024-07-18

arXiv.org

AI Summary

This paper addresses the challenge of evaluating time-series causal discovery algorithms by systematically investigating the sortability—specifically varsortability and R²-sortability—of autocorrelated stationary time series and its impact on causal inference performance. For the first time, these two sortability measures are extended to the stationary time-series setting. A multi-source evaluation framework is constructed, incorporating SVAR simulations, climate data, river runoff records, and synthetic data from the Causal Chamber generator, coupled with score-based causal discovery methods for empirical analysis. Results show that real-world time series typically exhibit high varsortability but low R²-sortability, indicating that variance structure encodes critical causal priors. Moreover, varsortability strongly and positively predicts causal discovery accuracy. This work introduces a quantifiable, interpretable evaluation dimension and a practical prior-based criterion for time-series causal inference.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Evaluating the performance of causal discovery algorithms that aim to find causal relationships between time-dependent processes remains a challenging topic. In this paper, we show that certain characteristics of datasets, such as varsortability (Reisach et al. 2021) and $R^2$-sortability (Reisach et al. 2023), also occur in datasets for autocorrelated stationary time series. We illustrate this empirically using four types of data: simulated data based on SVAR models and ErdH{o}s-R'enyi graphs, the data used in the 2019 causality-for-climate challenge (Runge et al. 2019), real-world river stream datasets, and real-world data generated by the Causal Chamber of (Gamella et al. 2024). To do this, we adapt var- and $R^2$-sortability to time series data. We also investigate the extent to which the performance of score-based causal discovery methods goes hand in hand with high sortability. Arguably, our most surprising finding is that the investigated real-world datasets exhibit high varsortability and low $R^2$-sortability indicating that scales may carry a significant amount of causal information.

Problem

• Evaluating causal discovery algorithms for time-dependent processes
• Assessing sortability characteristics in autocorrelated time series data
• Investigating performance link between score-based methods and sortability

Innovation

• Adapt var-sortability to time series
• Adapt R2-sortability to time series
• Evaluate score-based causal discovery methods