Data Transformation Strategies to Remove Heterogeneity

2025-07-16

AI Summary

Data heterogeneity—particularly format discrepancies—severely impedes AI model training efficiency and generalization. While existing research predominantly addresses structural or schema-level conflicts, it largely overlooks format-aware data transformation mechanisms. This paper presents the first systematic taxonomy of format-centric data transformation techniques, categorizing them into three classes: AI-input adaptation, format standardization, and structured/unstructured data conversion. Through a comprehensive literature review and comparative analysis, we delineate the applicability and limitations of each strategy. We further propose a transformation-centric data preprocessing framework that emphasizes semantic preservation and enhanced model adaptability. Our work fills a critical gap by providing the first unified survey of modern data transformation methodologies, offering both theoretical foundations and practical guidelines for mitigating the adverse effects of data heterogeneity. (149 words)

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Other Foundations of Data Mining & Knowledge Management
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Data heterogeneity is a prevalent issue, stemming from various conflicting factors, making its utilization complex. This uncertainty, particularly resulting from disparities in data formats, frequently necessitates the involvement of experts to find resolutions. Current methodologies primarily address conflicts related to data structures and schemas, often overlooking the pivotal role played by data transformation. As the utilization of artificial intelligence (AI) continues to expand, there is a growing demand for a more streamlined data preparation process, and data transformation becomes paramount. It customizes training data to enhance AI learning efficiency and adapts input formats to suit diverse AI models. Selecting an appropriate transformation technique is paramount in preserving crucial data details. Despite the widespread integration of AI across various industries, comprehensive reviews concerning contemporary data transformation approaches are scarce. This survey explores the intricacies of data heterogeneity and its underlying sources. It systematically categorizes and presents strategies to address heterogeneity stemming from differences in data formats, shedding light on the inherent challenges associated with each strategy.

Problem

• Addressing data heterogeneity from diverse conflicting sources
• Optimizing data transformation for AI model compatibility
• Reviewing strategies to resolve format disparities effectively

Innovation

• AI-driven data transformation enhances learning efficiency
• Systematic strategies address data format heterogeneity
• Customized input formats for diverse AI models