ZENN: A Thermodynamics-Inspired Computational Framework for Heterogeneous Data-Driven Modeling

2025-05-14

AI Summary

Modeling heterogeneous, multi-source data requires jointly learning structural representations and variability—a longstanding challenge. Method: This paper proposes ZENN, a thermodynamics-inspired framework that introduces *z-entropy*—a novel intrinsic entropy measure capturing fundamental data disparities—and establishes the first energy–entropy co-learning paradigm, moving beyond conventional entropy-based losses used solely as supervision signals. ZENN reconstructs neural architectures via differentiable, physics-constrained energy landscape modeling, higher-order derivative–based robust prediction, and DFT-driven Helmholtz free energy reconstruction. Results: ZENN significantly improves generalization and robustness on classification and energy landscape reconstruction tasks. It successfully reproduces the negative thermal expansion behavior of Fe₃Pt and accurately predicts critical points in temperature–pressure phase diagrams—outperforming state-of-the-art models in predictive accuracy.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Traditional entropy-based methods - such as cross-entropy loss in classification problems - have long been essential tools for quantifying uncertainty and disorder in data and developing artificial intelligence algorithms. However, the rapid growth of data across various domains has introduced new challenges, particularly the integration of heterogeneous datasets with intrinsic disparities. In this paper, we extend zentropy theory into the data science domain by introducing intrinsic entropy, enabling more effective learning from heterogeneous data sources. We propose a zentropy-enhanced neural network (ZENN) that simultaneously learns both energy and intrinsic entropy components, capturing the underlying structure of multi-source data. To support this, we redesign the neural network architecture to better reflect the intrinsic properties and variability inherent in diverse datasets. We demonstrate the effectiveness of ZENN on classification tasks and energy landscape reconstructions, showing its superior generalization capabilities and robustness-particularly in predicting high-order derivatives. As a practical application, we employ ZENN to reconstruct the Helmholtz energy landscape of Fe3Pt using data generated from DFT and capture key material behaviors, including negative thermal expansion and the critical point in the temperature-pressure space. Overall, our study introduces a novel approach for data-driven machine learning grounded in zentropy theory, highlighting ZENN as a versatile and robust deep learning framework for scientific problems involving complex, heterogeneous datasets.

Problem

• Integrating heterogeneous datasets with intrinsic disparities
• Learning from multi-source data using intrinsic entropy
• Reconstructing energy landscapes for complex material behaviors

Innovation

• Extends zentropy theory with intrinsic entropy
• Proposes ZENN for energy and entropy learning
• Redesigns neural network for heterogeneous datasets