data quality and integrity

In highly regulated domains such as finance, data quality control (QC) is often fragmented into isolated preprocessing steps, undermining end-to-end trustworthy AI pipelines. To address this, we propose the first AI-driven DataOps framework that embeds QC as a system-level core component. Our framework deeply integrates rule-based engines, statistical analysis, and custom AI-powered anomaly detection across the entire data lifecycle—from ingestion and transformation to model deployment—enabling dynamic remediation, policy-configurable workflows, and end-to-end auditability. Technically, it unifies data profiling, stream processing, cloud-native storage interfaces, and a proprietary AI detection module. Evaluated in a real-world financial production environment, the framework achieves significantly improved anomaly recall, reduces manual intervention by 42%, and ensures audit completeness and full data traceability under high-throughput conditions, fully satisfying regulatory compliance requirements.

This study addresses the lack of systematic evaluation of data quality tools with respect to their measurement capabilities and integration with large language models (LLMs). It presents the first multidimensional assessment framework grounded in real-world enterprise use cases, systematically evaluating six prominent tools—including open-source solutions such as Great Expectations and Deequ, as well as commercial platforms like Informatica and Experian—across dimensions including rule definition, duplicate detection, metric aggregation, and uncertainty handling, along with their LLM integration mechanisms. The findings reveal that commercial tools offer more comprehensive functionality and初步 support for LLM-assisted rule generation, whereas open-source tools provide greater flexibility at the cost of higher implementation effort. Notably, none of the evaluated tools currently enable direct LLM-based data validation. This work provides empirical guidance for selecting data quality tools and advancing their integration with LLMs.

The core challenge in data quality monitoring lies in error provenance—specifically, identifying the underlying mechanisms that generate errors—a problem largely overlooked by existing work, which seldom models such mechanisms explicitly. This paper focuses on errors arising from intrinsic dependencies within data and proposes MechDetect, the first method to systematically extend missing-data mechanism detection to diverse error types—including outliers, inconsistencies, and format violations. Leveraging joint statistical modeling and supervised learning, MechDetect simultaneously models tabular data and their error masks to automatically determine whether observed errors stem from inherent characteristics of the original data. Extensive experiments across multiple benchmark datasets demonstrate that MechDetect significantly outperforms state-of-the-art baselines in accurately diagnosing error-generation mechanisms. By providing mechanistic interpretability, it establishes a theoretical foundation and practical framework for explainable data repair.

This study addresses the compliance challenges faced by data practitioners in machine learning systems under regulations such as the GDPR and the AI Act, particularly concerning data quality. Through semi-structured interviews with practitioners in the European Union, combined with thematic analysis of regulatory texts and engineering workflows, the research systematically uncovers a structural disconnect between regulation-driven data quality requirements and ML engineering practices. It identifies five core challenges: misalignment between legal principles and engineering implementation, fragmented data pipelines, lack of purpose-built compliance tools, ambiguous accountability, and reactive responses to audits. Building on these findings, the work proposes directions for designing compliance-oriented tooling, establishing effective governance mechanisms, and fostering cultural transformation to bridge the gap between regulatory mandates and practical ML development.

This work addresses the challenge of enforcing temporal safety constraints throughout the lifecycle of black-box AI systems, such as large language models (LLMs), which are inherently difficult to verify. The paper proposes the first offline auditing and online monitoring framework that integrates Linear Temporal Logic (LTL) with machine learning. This framework enables formal verification of complex temporal behavioral specifications by introducing a sampling-driven predictive monitor and an intervenable runtime monitor, effectively preventing policy violations. Experimental results demonstrate that the proposed approach significantly outperforms existing LLM-based evaluators in detecting temporal violations. Notably, it achieves performance on par with or superior to state-of-the-art large models using only a small labeled model, while its intervention mechanism substantially reduces violation rates without compromising task performance.

Ad hoc SQL development lacks engineering rigor, leading to data silos, logical redundancy, and ineffective data governance. Method: This paper proposes a DataOps-driven CI/CD framework for analytical SQL warehouses, featuring a novel five-stage automated pipeline—Lint, Optimize, Parse, Validate, Observe—that embeds quality assurance and enables end-to-end lifecycle governance. Contribution/Results: We introduce the DataOps Controls Scorecard and a requirements traceability matrix, explicitly mapping 12 governance criteria to CI/CD stages to ensure control completeness and scalability. The framework integrates Agile, Lean, and DevOps principles with static analysis, syntactic parsing, optimization recommendations, validation testing, and observability. Empirical evaluation demonstrates significant improvements in data quality, development transparency, and cross-functional collaboration, providing a sustainable, production-ready pathway for large-scale analytical systems.

Existing approaches struggle to effectively quantify the similarity and quality between synthetic and real data in evaluating tool-augmented agents. To address this gap, this work proposes SynAE, a novel framework that establishes the first multi-axis evaluation system tailored for multi-turn tool-use scenarios. SynAE introduces four fine-grained metric categories—assessing task instructions, tool invocations, final outputs, and downstream evaluation performance—to systematically measure synthetic data across dimensions of validity, fidelity, and diversity. Integrating natural language processing, trajectory modeling, and controllable generation techniques, the framework enables a reproducible evaluation pipeline and successfully identifies several representative failure modes in synthetic data generation. Empirical results demonstrate that such multidimensional assessment is essential for enhancing the reliability of agent evaluations.

Existing database systems struggle to effectively model temporal dynamics, contextual dependencies, and causal relationships among attributes. To address this limitation, this work proposes Change Rules (CRs)—a novel rule-based paradigm that explicitly captures antecedent-consequent attribute changes within ordered tuple sequences, thereby overcoming the constraints of traditional data quality rules in modeling temporal and contextual patterns. The authors introduce CR-Miner, an efficient algorithm that employs a level-wise candidate generation strategy to identify change intervals, integrating declarative dependency specifications with sequence analysis techniques. Experimental results demonstrate that CR-Miner achieves a 40–50% average speedup over state-of-the-art methods while significantly enhancing the granularity and efficiency of trend analysis and causal inference.

Current evaluations of AI systems predominantly rely on static benchmarks, which fail to capture behavioral risks in dynamic real-world environments. This work formalizes AI auditing as an uncertainty-aware, dynamic constraint monitoring problem across the system’s entire lifecycle, targeting critical attributes such as fairness and safety while integrating sociotechnical norms with statistical risk control. By developing a theoretical framework and supporting infrastructure for continuous auditing, the study advances AI governance beyond one-off testing toward ongoing, reliable, and accountable oversight mechanisms.

This study addresses the challenges posed by rapid evolution in digital forensic systems and tools, which induces drift in evidentiary behaviors and tool outputs, thereby undermining result reproducibility and trustworthiness. To mitigate this, the authors propose a test-driven forensic methodology that introduces state-transition testing for causal attribution, encoding forensic expectations as executable specifications. The approach integrates virtual machine environments with computer vision–guided GUI automation to simulate authentic user interactions and verify system state changes. An open web platform is developed to facilitate sharing and replication of experiments. The method’s efficacy is demonstrated through five case studies, including a regression analysis across 25 versions of Autopsy, which uncovered numerous undocumented, substantial changes in its reporting output.