Multi-Modal Attention Networks for Enhanced Segmentation and Depth Estimation of Subsurface Defects in Pulse Thermography

2025-01-17

AI Summary

In pulsed thermography (PTI) non-destructive testing, conventional principal component analysis (PCA) and thermal signal reconstruction (TSR) feature representations suffer from inadequate subsurface defect segmentation and depth estimation due to their decoupled, unimodal nature. To address this, we propose PT-Fusion, a multimodal attention-based fusion network. PT-Fusion introduces two novel components: the Encoder Attention Fusion Gate (EAFG), enabling adaptive weighted fusion of PCA and TSR features within the encoder, and the Attention-Enhanced Decoder Block (AEDB), facilitating cross-modal interaction and refinement in the decoder. Additionally, a thermal sequence random sampling data augmentation strategy is incorporated to mitigate small-sample limitations. Built upon a U-Net variant architecture, PT-Fusion achieves ~10% improvement in both defect segmentation and depth estimation accuracy over U-Net, Attention U-Net, and 3D-CNN on the Laval IRT-PVC dataset, significantly enhancing localization precision and quantitative assessment capability for internal defects in industrial components.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

AI-driven pulse thermography (PT) has become a crucial tool in non-destructive testing (NDT), enabling automatic detection of hidden anomalies in various industrial components. Current state-of-the-art techniques feed segmentation and depth estimation networks compressed PT sequences using either Principal Component Analysis (PCA) or Thermographic Signal Reconstruction (TSR). However, treating these two modalities independently constrains the performance of PT inspection models as these representations possess complementary semantic features. To address this limitation, this work proposes PT-Fusion, a multi-modal attention-based fusion network that fuses both PCA and TSR modalities for defect segmentation and depth estimation of subsurface defects in PT setups. PT-Fusion introduces novel feature fusion modules, Encoder Attention Fusion Gate (EAFG) and Attention Enhanced Decoding Block (AEDB), to fuse PCA and TSR features for enhanced segmentation and depth estimation of subsurface defects. In addition, a novel data augmentation technique is proposed based on random data sampling from thermographic sequences to alleviate the scarcity of PT datasets. The proposed method is benchmarked against state-of-the-art PT inspection models, including U-Net, attention U-Net, and 3D-CNN on the Universit'e Laval IRT-PVC dataset. The results demonstrate that PT-Fusion outperforms the aforementioned models in defect segmentation and depth estimation accuracies with a margin of 10%.

Problem

• Pulse Thermography
• Image Processing
• Non-Destructive Testing

Innovation

• PT-Fusion
• Attention-based Feature Fusion
• Random Heat Image Selection Strategy