AI Data Readiness Inspector (AIDRIN) for Quantitative Assessment of Data Readiness for AI

2024-06-27

International Conference on Statistical and Scientific Database Management

AI Summary

Existing AI applications lack a unified framework for quantitatively assessing Data Readiness for AI, hindering the integrated diagnosis of data quality and AI suitability. Method: We propose AIDRIN—the first quantitative, multidimensional framework for AI-oriented data readiness evaluation—systematically integrating traditional data quality dimensions (e.g., completeness, consistency) with AI-specific metrics (e.g., class imbalance, feature importance, algorithmic fairness, privacy risk, and FAIR compliance). AIDRIN unifies data quality analysis, statistical modeling, fairness auditing, privacy risk assessment, and FAIR validation, supported by interactive visualization. Contribution/Results: Evaluated across multiple real-world scenarios, AIDRIN reduces AI readiness assessment time by over 40%, significantly accelerating preprocessing decisions and enabling robust, interpretable, and trustworthy machine learning deployment.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Humans and AI: AI for Accessibility
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

"Garbage In Garbage Out"is a universally agreed quote by computer scientists from various domains, including Artificial Intelligence (AI). As data is the fuel for AI, models trained on low-quality, biased data are often ineffective. Computer scientists who use AI invest a considerable amount of time and effort in preparing the data for AI. However, there are no standard methods or frameworks for assessing the"readiness"of data for AI. To provide a quantifiable assessment of the readiness of data for AI processes, we define parameters of AI data readiness and introduce AIDRIN (AI Data Readiness Inspector). AIDRIN is a framework covering a broad range of readiness dimensions available in the literature that aid in evaluating the readiness of data quantitatively and qualitatively. AIDRIN uses metrics in traditional data quality assessment such as completeness, outliers, and duplicates for data evaluation. Furthermore, AIDRIN uses metrics specific to assess data for AI, such as feature importance, feature correlations, class imbalance, fairness, privacy, and FAIR (Findability, Accessibility, Interoperability, and Reusability) principle compliance. AIDRIN provides visualizations and reports to assist data scientists in further investigating the readiness of data. The AIDRIN framework enhances the efficiency of the machine learning pipeline to make informed decisions on data readiness for AI applications.

Problem

• Lack of standard methods for assessing AI data readiness.
• Need for quantifiable evaluation of data quality for AI processes.
• Framework to enhance efficiency in machine learning data preparation.

Innovation

• AIDRIN framework quantifies AI data readiness.
• Uses traditional and AI-specific data quality metrics.
• Provides visualizations and reports for data assessment.