Sanity Checks for Agentic Data Science

2026-04-13

AI Summary

Agentic data science systems often produce deceptively optimistic conclusions that are difficult to detect and lack robust mechanisms for reliability validation. This work addresses this challenge by introducing falsifiability constraints into the domain for the first time, leveraging the Predictability-Computability-Stability (PCS) framework to propose a lightweight dual-check mechanism. The approach evaluates an agent’s ability to distinguish signal from noise through controlled perturbations, thereby quantifying the stability of its conclusions. Experiments across 11 real-world datasets reveal that in 6 of them, the affirmative conclusions generated by agentic systems lack empirical support, and their self-reported confidence levels exhibit significant misalignment with actual stability. These findings underscore both the necessity and effectiveness of the proposed method.

Technology Category

• Multiagent Systems: Other Foundations of Multi Agent Systems
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Agentic data science (ADS) pipelines have grown rapidly in both capability and adoption, with systems such as OpenAI Codex now able to directly analyze datasets and produce answers to statistical questions. However, these systems can reach falsely optimistic conclusions that are difficult for users to detect. To address this, we propose a pair of lightweight sanity checks grounded in the Predictability-Computability-Stability (PCS) framework for veridical data science. These checks use reasonable perturbations to screen whether an agent can reliably distinguish signal from noise, acting as a falsifiability constraint that can expose affirmative conclusions as unsupported. Together, the two checks characterize the trustworthiness of an ADS output, e.g. whether it has found stable signal, is responding to noise, or is sensitive to incidental aspects of the input. We validate the approach on synthetic data with controlled signal-to-noise ratios, confirming that the sanity checks track ground-truth signal strength. We then demonstrate the checks on 11 real-world datasets using OpenAI Codex, characterizing the trustworthiness of each conclusion and finding that in 6 of the datasets an affirmative conclusion is not well-supported, even though a single ADS run may support one. We further analyze failure modes of ADS systems and find that ADS self-reported confidence is poorly calibrated to the empirical stability of its conclusions.

Problem

• Agentic Data Science
• falsifiability
• signal-to-noise
• trustworthiness
• statistical conclusion

Innovation

• Agentic Data Science
• Sanity Checks
• Predictability-Computability-Stability (PCS)
• Signal-to-Noise Discrimination
• Falsifiability Constraint