TSRating: Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment

2025-06-02

AI Summary

Addressing the challenges of cross-domain time-series data quality assessment—namely, difficulty in reliable evaluation and poor generalization—this paper proposes TSRating, a unified framework. TSRating innovatively employs large language models (LLMs) as implicit quality annotators, leveraging prompt engineering to elicit their capability for discriminating quality across multivariate time-series samples. It further introduces TSRater, a lightweight meta-scorer that adapts rapidly across domains via a hypergradient-free signSGD-optimized variant of MAML, enabling efficient pairwise quality comparison without domain-specific feature engineering. The framework is inherently domain-agnostic. Extensive experiments on 11 benchmark datasets and three downstream tasks demonstrate that TSRating significantly outperforms state-of-the-art methods in accuracy, inference efficiency, and cross-domain generalization.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

High-quality time series (TS) data are essential for ensuring TS model performance, rendering research on rating TS data quality indispensable. Existing methods have shown promising rating accuracy within individual domains, primarily by extending data quality rating techniques such as influence functions and Shapley values to account for temporal characteristics. However, they neglect the fact that real-world TS data can span vastly different domains and exhibit distinct properties, hampering the accurate and efficient rating of diverse TS data. In this paper, we propose TSRating, a novel and unified framework for rating the quality of time series data crawled from diverse domains. TSRating is built on the assumption that LLMs inherit ample knowledge, acquired during their extensive pretraining, enabling them to comprehend and discern quality differences in diverse TS data. We verify this assumption by devising a series of prompts to elicit quality comparisons from LLMs for pairs of TS samples. We then fit a dedicated rating model, termed TSRater, to convert the LLMs' judgments into efficient quality predictions via TSRater's inference on future TS samples. To ensure cross-domain adaptability, we develop a meta-learning scheme to train TSRater on quality comparisons collected from nine distinct domains. To improve training efficiency, we employ signSGD for inner-loop updates, thus circumventing the demanding computation of hypergradients. Extensive experimental results on eleven benchmark datasets across three time series tasks, each using both conventional TS models and TS foundation models, demonstrate that TSRating outperforms baselines in terms of estimation accuracy, efficiency, and domain adaptability.

Problem

• Rating quality of diverse time series data across domains
• Leveraging LLM knowledge for accurate TS quality assessment
• Meta-learning framework for cross-domain TS quality prediction

Innovation

• Meta-learning from LLM judgments for TS quality
• Cross-domain adaptability via meta-learning scheme
• SignSGD for efficient inner-loop updates