There is No "apple" in Timeseries: Rethinking TSFM through the Lens of Invariance

2025-10-22

AI Summary

Existing time-series foundation models (TSFMs) underperform primarily due to the uncritical transfer of NLP/CV paradigms, which overlooks two intrinsic properties of time-series data: the absence of conceptual annotations and the impossibility of large-scale web crawling. To address this, we propose a data ontology framework grounded in **temporal semantic invariance** as a first principle—systematically encompassing translation, scaling, and amplitude/phase perturbations to ensure representation completeness. Our method integrates classical model priors with supervised signals to design a novel, semantics-preserving training paradigm. By abandoning opportunistic data aggregation, our approach achieves substantial improvements in generalization, reasoning capability, and emergent behaviors. It establishes a principled, interpretable, and scalable design pathway for next-generation TSFMs.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Timeseries foundation models (TSFMs) have multiplied, yet lightweight supervised baselines and even classical models often match them. We argue this gap stems from the naive importation of NLP or CV pipelines. In language and vision, large web-scale corpora densely capture human concepts i.e. there are countless images and text of apples. In contrast, timeseries data is built to complement the image and text modalities. There are no timeseries dataset that contains the concept apple. As a result, the scrape-everything-online paradigm fails for TS. We posit that progress demands a shift from opportunistic aggregation to principled design: constructing datasets that systematically span the space of invariance that preserve temporal semantics. To this end, we suggest that the ontology of timeseries invariances should be built based on first principles. Only by ensuring representational completeness through invariance coverage can TSFMs achieve the aligned structure necessary for generalisation, reasoning, and truly emergent behaviour.

Problem

• Rethinking timeseries foundation models due to poor performance versus baselines
• Addressing failure of web-scraping paradigm for timeseries data collection
• Proposing principled dataset design based on temporal invariance principles

Innovation

• Construct datasets spanning temporal invariance space
• Build ontology of timeseries invariances from first principles
• Ensure representational completeness through invariance coverage