AIDRIN 2.0: A Framework to Assess Data Readiness for AI

2025-05-22

AI Summary

This study addresses the absence of systematic data readiness assessment in AI applications by proposing the first multi-dimensional data governance evaluation framework, covering data quality, bias, fairness, and privacy. Methodologically, it innovatively integrates privacy-preserving federated learning (PPFL) to enable decentralized, cross-organizational assessment while preserving data sovereignty; it further introduces a user-friendly interface and an interpretable, multi-dimensional quantitative evaluation model. Experiments on real-world datasets demonstrate that the framework accurately identifies critical data readiness deficiencies impeding AI model performance and effectively guides data governance improvements. Key contributions include: (1) the first deep integration of PPFL into the data readiness assessment pipeline, enhancing scalability and trustworthiness; (2) significantly improved usability for non-technical stakeholders; and (3) practical support for cross-domain collaboration—thereby establishing an extensible, trustworthy AI deployment infrastructure.

Technology Category

• Humans and AI: AI for Accessibility
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

AI Data Readiness Inspector (AIDRIN) is a framework to evaluate and improve data preparedness for AI applications. It addresses critical data readiness dimensions such as data quality, bias, fairness, and privacy. This paper details enhancements to AIDRIN by focusing on user interface improvements and integration with a privacy-preserving federated learning (PPFL) framework. By refining the UI and enabling smooth integration with decentralized AI pipelines, AIDRIN becomes more accessible and practical for users with varying technical expertise. Integrating with an existing PPFL framework ensures that data readiness and privacy are prioritized in federated learning environments. A case study involving a real-world dataset demonstrates AIDRIN's practical value in identifying data readiness issues that impact AI model performance.

Problem

• Evaluates and improves AI data readiness dimensions
• Enhances UI and integrates privacy-preserving federated learning
• Identifies data issues impacting AI model performance

Innovation

• Evaluates data quality, bias, fairness, privacy
• Enhances UI for user accessibility
• Integrates with privacy-preserving federated learning