Generative Multimodal Multiscale Data Fusion for Digital Twins in Aerosol Jet Electronics Printing

2025-04-30

AI Summary

Aerosol jet printing (AJP) faces challenges in fusing heterogeneous multimodal (optical microscopy/ confocal profilometry) and multiscale sensing data, while existing process–structure–property (PSP) models suffer from limited interpretability and generalizability. Method: This work pioneers the integration of diffusion models into AJP PSP modeling, proposing a two-stage generative framework—“registration-then-fusion”: first explicitly modeling cross-modal causal relationships via multimodal feature registration, then achieving high-fidelity optical microscopy (OM) and confocal profilometry (CP) data fusion through cross-scale feature alignment. Contribution/Results: The approach overcomes long-standing bottlenecks in heterogeneous data fusion, significantly improving PSP relationship reconstruction accuracy. It establishes an interpretable, generalizable, generative modeling foundation for AJP digital twins, enabling robust inference across diverse material systems and process conditions.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The rising demand for high-value electronics necessitates advanced manufacturing techniques capable of meeting stringent specifications for precise, complex, and compact devices, driving the shift toward innovative additive manufacturing (AM) solutions. Aerosol Jet Printing (AJP) is a versatile AM technique that utilizes aerosolized functional materials to accurately print intricate patterns onto diverse substrates. Machine learning (ML)- based Process-Structure-Property (PSP) modeling is essential for enhancing AJP manufacturing, as it quantitatively connects process parameters, structural features, and resulting material properties. However, current ML approaches for modeling PSP relationships in AJP face significant limitations in handling multimodal and multiscale data, underscoring a critical need for generative methods capable of comprehensive analysis through multimodal and multiscale fusion. To address this challenge, this study introduces a novel generative modeling methodology leveraging diffusion models for PSP data fusion in AJP. The proposed method integrates multimodal, multiscale PSP features in two phases: (1) registering the features, and (2) fusing them to generate causal relationships between PSP attributes. A case study demonstrates the registration and fusion of optical microscopy (OM) images and confocal profilometry (CP) data from AJP, along with the fine-tuning of the fusion step. The results effectively capture complex PSP relationships, offering deeper insights into digital twins of dynamic manufacturing systems.

Problem

• Handling multimodal multiscale data in Aerosol Jet Printing
• Enhancing Process-Structure-Property modeling with generative methods
• Fusing optical microscopy and confocal profilometry data effectively

Innovation

• Generative multimodal multiscale data fusion
• Diffusion models for PSP relationships
• Integration of optical and profilometry data