DataPro - A Standardized Data Understanding and Processing Procedure: A Case Study of an Eco-Driving Project

2025-01-21

EI.A

AI Summary

To address modeling bias arising from semantic ambiguity, missing annotations, and heterogeneous quality in multi-source vehicular data for eco-driving, this paper proposes a domain-knowledge-embedded standardized data engineering framework. Methodologically, we establish a hierarchical data maturity assessment system and design a reusable four-stage pipeline—comprising understanding, cleaning, augmentation, and alignment—that integrates rule-based cleaning, lightweight active learning for annotation, spatiotemporal consistency verification, and driving-behavior-graph-guided data augmentation. Our key contribution lies in the first explicit standardization of the data engineering process and its deep coupling with traffic-domain constraints. Evaluated on real-world fleet data, the framework improves prediction accuracy of downstream energy-saving strategy models by 12.7% and reduces data preparation time by 64%. The pipeline has been successfully reused across three traffic AI projects.

Technology Category

• Application Domains: Mobility, Driving & Flight
• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Knowledge Representation and Reasoning: Knowledge Engineering

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Problem

• Data Handling
• Interdepartmental Collaboration
• Business Decision Support

Innovation

• DataPro
• CRISP-DM improvement
• Technical Understanding and Implementation