AutoDataset: A Lightweight System for Continuous Dataset Discovery and Search

2026-03-07

AI Summary

Existing dataset discovery platforms rely on manual curation, resulting in limited coverage and delayed updates. To address this, this work proposes a “paper-first” automated paradigm and implements a lightweight system that continuously monitors arXiv to enable end-to-end, low-latency dataset extraction and dense semantic retrieval. The pipeline leverages a lightweight classifier, GROBID-based parsing, sentence-level description extraction, and a LaTeX source fallback mechanism. The system achieves an inference latency of only 11 ms and an F1 score of 0.94, improving dataset discovery efficiency by 80%. It has been deployed as a continuously updated online service, significantly accelerating researchers’ access to and retrieval of newly introduced datasets.

Technology Category

• Natural Language Processing: Information Extraction
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The continuous expansion of task-specific datasets has become a major driver of progress in machine learning. However, discovering newly released datasets remains difficult, as existing platforms largely depend on manual curation or community submissions, leading to limited coverage and substantial delays. To address this challenge, we introduce AutoDataset, a lightweight, automated system for real-time dataset discovery and retrieval. AutoDataset adopts a paper-first approach by continuously monitoring arXiv to detect and index datasets directly from newly published research. The system operates through a low-overhead multi-stage pipeline. First, a lightweight classifier rapidly filters titles and abstracts to identify papers releasing datasets, achieving an F1 score of 0.94 with an inference latency of 11 ms. For identified papers, we parse PDFs with GROBID and apply a sentence-level extractor to extract dataset descriptions. Dataset URLs are extracted from the paper text with an automated fallback to LaTeX source analysis when needed. Finally, the structured records are indexed using a dense semantic retriever, enabling low-latency natural language search. We deploy AutoDataset as a live system that continuously ingests new papers and provides up-to-date dataset discovery. In practice, it has been shown to significantly reduce the time required for researchers to locate newly released datasets, improving dataset discovery efficiency by up to 80%.

Problem

• dataset discovery
• machine learning
• arXiv
• real-time search
• data curation

Innovation

• AutoDataset
• dataset discovery
• real-time indexing
• semantic retrieval
• lightweight pipeline