🤖 AI Summary
This study addresses the widespread lack of compliance disclosures regarding data privacy regulations—such as GDPR and CCPA—in AI datasets hosted on the Hugging Face platform, which poses significant regulatory risks. For the first time, the paper introduces “compliance readiness” as a core dimension of dataset quality and systematically evaluates transparency in data provenance, processing workflows, and handling of sensitive information. Through an analysis of 11,682 public datasets—combining automated text mining of dataset cards with manual sampling and review—the research reveals that the vast majority fail to clearly document their creation processes or data sources, and only a small fraction explicitly identify sensitive fields such as personally identifiable information (PII). These findings highlight critical gaps in current practices and provide an empirical foundation for developing standardized tools and benchmarks for assessing dataset compliance.
📝 Abstract
A very large number of datasets are made available via Hugging Face (HF). These datasets are used to train a vast number of AI models, also available via HF. There are regulatory issues that might arise in the AI supply chain when those datasets do not follow, for instance, data privacy practices, or do not disclose their data sources, cleaning and preparation processes. In this preliminary work, regulatory compliance-readiness is examined as a quality attribute in the framework of HF datasets, with a main focus on data privacy (e.g. GDPR, CCPA). Towards this direction, an analysis on regulatory compliance (e.g. with privacy laws) of the dataset using the dataset card and the dataset structure overview as starting points has been performed. We collected 11,682 HF datasets that have a dataset card, are not gated and have at least 500 downloads and analyzed the datasets using automated techniques and manual analysis on a sample of the dataset. The results show that a very small number of datasets are explicit on the datasets creation processes and mention data sources, while some make mentions to Personally Identifiable Information (PII) or other sensitive data in their data schemas. These results show the need for more detailed examinations of regulatory compliance in AI datasets and models, and research on tools that provide guidance to practitioners and automate the compliance process.