Cross-Modal Mapping and Dual-Branch Reconstruction for 2D-3D Multimodal Industrial Anomaly Detection

2026-03-04

AI Summary

This work addresses the challenge of insufficient robustness in unsupervised industrial anomaly detection caused by sparse depth, low-texture regions, or missing modalities. To this end, we propose CMDR-IAD, a lightweight and modality-flexible unsupervised framework that models appearance-geometry consistency through bidirectional 2D↔3D cross-modal mapping. The method employs dual-branch independent reconstruction to separately capture normal texture and geometric structures, and introduces a reliability gating mechanism with confidence-weighted fusion to effectively handle scenarios with sparse depth or minimal texture. Notably, CMDR-IAD achieves precise anomaly localization without requiring a memory bank, operating seamlessly in both single- and multi-modal settings. On the MVTec 3D-AD benchmark, it attains 97.3% image-level AUROC, 99.6% pixel-level AUROC, and 97.6% AUPRO; under 3D-only mode on a real-world polyurethane cutting dataset, it achieves 92.6% I-AUROC and 92.5% P-AUROC.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Multimodal industrial anomaly detection benefits from integrating RGB appearance with 3D surface geometry, yet existing \emph{unsupervised} approaches commonly rely on memory banks, teacher-student architectures, or fragile fusion schemes, limiting robustness under noisy depth, weak texture, or missing modalities. This paper introduces \textbf{CMDR-IAD}, a lightweight and modality-flexible unsupervised framework for reliable anomaly detection in 2D+3D multimodal as well as single-modality (2D-only or 3D-only) settings. \textbf{CMDR-IAD} combines bidirectional 2D$\leftrightarrow$3D cross-modal mapping to model appearance-geometry consistency with dual-branch reconstruction that independently captures normal texture and geometric structure. A two-part fusion strategy integrates these cues: a reliability-gated mapping anomaly highlights spatially consistent texture-geometry discrepancies, while a confidence-weighted reconstruction anomaly adaptively balances appearance and geometric deviations, yielding stable and precise anomaly localization even in depth-sparse or low-texture regions. On the MVTec 3D-AD benchmark, CMDR-IAD achieves state-of-the-art performance while operating without memory banks, reaching 97.3\% image-level AUROC (I-AUROC), 99.6\% pixel-level AUROC (P-AUROC), and 97.6\% AUPRO. On a real-world polyurethane cutting dataset, the 3D-only variant attains 92.6\% I-AUROC and 92.5\% P-AUROC, demonstrating strong effectiveness under practical industrial conditions. These results highlight the framework's robustness, modality flexibility, and the effectiveness of the proposed fusion strategies for industrial visual inspection. Our source code is available at https://github.com/ECGAI-Research/CMDR-IAD/

Problem

• multimodal anomaly detection
• unsupervised learning
• 2D-3D fusion
• industrial inspection
• cross-modal consistency

Innovation

• cross-modal mapping
• dual-branch reconstruction
• unsupervised anomaly detection
• modality flexibility
• reliability-gated fusion