🤖 AI Summary
Web-scraped datasets commonly suffer from low quality, high redundancy, and class imbalance, rendering existing heuristic filtering methods inadequate for modeling complex multimodal features—often introducing bias or erroneously discarding relevant samples. To address this, we propose EcoDatum, the first quality-centric multimodal collaborative filtering framework. Its core innovations are: (1) a quality-guided multimodal deduplication mechanism that jointly leverages visual, linguistic, and cross-modal embeddings for fine-grained similarity assessment; and (2) a weakly supervised ensemble optimization framework integrating automated hyperparameter search with multi-operator collaborative scoring. Evaluated on the DataComp benchmark, EcoDatum achieves a mean score of 0.182—outperforming prior baselines by 28%—and ranks first overall. Empirical results demonstrate substantial improvements in downstream model training efficiency and generalization performance.
📝 Abstract
In an era overwhelmed by vast amounts of data, the effective curation of web-crawl datasets is essential for optimizing model performance. This paper tackles the challenges associated with the unstructured and heterogeneous nature of such datasets. Traditional heuristic curation methods often inadequately capture complex features, resulting in biases and the exclusion of relevant data. We introduce an advanced, learning-driven approach, Ensemble Curation Of DAta ThroUgh Multimodal Operators (EcoDatum), incorporating a novel quality-guided deduplication method to ensure balanced feature distributions. EcoDatum strategically integrates various unimodal and multimodal data curation operators within a weak supervision ensemble framework, utilizing automated optimization to score each data point effectively. EcoDatum, which significantly improves the data curation quality and efficiency, outperforms existing state-of-the-art (SOTA) techniques, ranked 1st on the DataComp leaderboard, with an average performance score of 0.182 across 38 diverse evaluation datasets. This represents a 28% improvement over the DataComp baseline method, demonstrating its effectiveness in improving dataset curation and model training efficiency.