Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks

2025-02-14

AI Summary

Data scientists and domain experts face inefficient collaboration in time-series forecasting due to the technical complexity and poor interpretability of model specification, leading to misalignment between model behavior and decision-making intent. Method: This paper proposes an interactive, human-AI collaborative modeling support framework. Contributions/Results: (1) We introduce the first explainable time-series query language designed for non-technical users, enabling natural-language specification of forecasting objectives and constraints; (2) we develop a specification-transparent prototyping mechanism and a subgroup-aware model evaluation framework for fine-grained behavioral validation; (3) we integrate an interactive visualization system to support iterative specification refinement and rapid feedback. Evaluated on three real-world industrial case studies, our approach reduces specification screening time by 70% on average, significantly improving efficiency in identifying appropriate modeling directions and verifying solution feasibility.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search

Abstract

Temporal predictive models have the potential to improve decisions in health care, public services, and other domains, yet they often fail to effectively support decision-makers. Prior literature shows that many misalignments between model behavior and decision-makers' expectations stem from issues of model specification, namely how, when, and for whom predictions are made. However, model specifications for predictive tasks are highly technical and difficult for non-data-scientist stakeholders to interpret and critique. To address this challenge we developed Tempo, an interactive system that helps data scientists and domain experts collaboratively iterate on model specifications. Using Tempo's simple yet precise temporal query language, data scientists can quickly prototype specifications with greater transparency about pre-processing choices. Moreover, domain experts can assess performance within data subgroups to validate that models behave as expected. Through three case studies, we demonstrate how Tempo helps multidisciplinary teams quickly prune infeasible specifications and identify more promising directions to explore.

Problem

• Improves collaborative model specification
• Addresses misalignment in temporal predictions
• Enhances model transparency and validation

Innovation

• Interactive collaborative system
• Temporal query language
• Multidisciplinary team validation