OpenConstruction: A Systematic Synthesis of Open Visual Datasets for Data-Centric Artificial Intelligence in Construction Monitoring

2025-08-15

AI Summary

Current publicly available computer vision datasets in the construction domain suffer from significant limitations in scale, modality diversity, annotation quality, and representativeness of real-world scenarios; moreover, they lack a systematic classification and evaluation framework, hindering the reliability and scalability of AI applications. To address this, we conduct a comprehensive survey of 51 publicly accessible construction-related vision datasets published between 2005 and 2024, encompassing RGB images, videos, and point clouds. We propose the first FAIR (Findable, Accessible, Interoperable, Reusable)-oriented classification framework tailored for construction monitoring, featuring a structured schema that enables multi-dimensional annotation characterization and application mapping. Furthermore, we develop and open-source OpenConstruction—a standardized, reproducible data catalog supporting key intelligent construction tasks such as progress tracking and safety monitoring. This work advances a data-centric paradigm for AI in construction.

Technology Category

• Computer Vision: Applications
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Feature Construction/Reformulation

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

The construction industry increasingly relies on visual data to support Artificial Intelligence (AI) and Machine Learning (ML) applications for site monitoring. High-quality, domain-specific datasets, comprising images, videos, and point clouds, capture site geometry and spatiotemporal dynamics, including the location and interaction of objects, workers, and materials. However, despite growing interest in leveraging visual datasets, existing resources vary widely in sizes, data modalities, annotation quality, and representativeness of real-world construction conditions. A systematic review to categorize their data characteristics and application contexts is still lacking, limiting the community's ability to fully understand the dataset landscape, identify critical gaps, and guide future directions toward more effective, reliable, and scalable AI applications in construction. To address this gap, this study conducts an extensive search of academic databases and open-data platforms, yielding 51 publicly available visual datasets that span the 2005-2024 period. These datasets are categorized using a structured data schema covering (i) data fundamentals (e.g., size and license), (ii) data modalities (e.g., RGB and point cloud), (iii) annotation frameworks (e.g., bounding boxes), and (iv) downstream application domains (e.g., progress tracking). This study synthesizes these findings into an open-source catalog, OpenConstruction, supporting data-driven method development. Furthermore, the study discusses several critical limitations in the existing construction dataset landscape and presents a roadmap for future data infrastructure anchored in the Findability, Accessibility, Interoperability, and Reusability (FAIR) principles. By reviewing the current landscape and outlining strategic priorities, this study supports the advancement of data-centric solutions in the construction sector.

Problem

• Lack of systematic review for construction visual datasets
• Variability in dataset quality and representativeness limits AI applications
• Need for structured categorization and FAIR-compliant data infrastructure

Innovation

• Systematic review of 51 open visual datasets
• Categorization using structured data schema
• Open-source catalog for data-driven development