NSF-MAP: Neurosymbolic Multimodal Fusion for Robust and Interpretable Anomaly Prediction in Assembly Pipelines

2025-05-09

AI Summary

To address the insufficient robustness and interpretability of anomaly prediction for multi-source heterogeneous data (time-series sensor signals and industrial images) in assembly lines, this paper proposes a neuro-symbolic fusion framework. The method introduces a novel decision-level temporal–image fusion modeling approach, integrating fusion-oriented transfer learning with knowledge-graph-guided knowledge injection learning to synergize neural perception and symbolic reasoning. It enables real-time defect identification and achieves a 12.7% improvement in anomaly detection accuracy and a 34% reduction in false positive rate over unimodal baselines on both proprietary and public multimodal datasets. Key contributions are: (1) the first decision-level multimodal neuro-symbolic fusion paradigm specifically designed for assembly-line scenarios; and (2) an interpretable modeling mechanism that ensures both industrial-grade robustness and decision traceability.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In modern assembly pipelines, identifying anomalies is crucial in ensuring product quality and operational efficiency. Conventional single-modality methods fail to capture the intricate relationships required for precise anomaly prediction in complex predictive environments with abundant data and multiple modalities. This paper proposes a neurosymbolic AI and fusion-based approach for multimodal anomaly prediction in assembly pipelines. We introduce a time series and image-based fusion model that leverages decision-level fusion techniques. Our research builds upon three primary novel approaches in multimodal learning: time series and image-based decision-level fusion modeling, transfer learning for fusion, and knowledge-infused learning. We evaluate the novel method using our derived and publicly available multimodal dataset and conduct comprehensive ablation studies to assess the impact of our preprocessing techniques and fusion model compared to traditional baselines. The results demonstrate that a neurosymbolic AI-based fusion approach that uses transfer learning can effectively harness the complementary strengths of time series and image data, offering a robust and interpretable approach for anomaly prediction in assembly pipelines with enhanced performance. oindent The datasets, codes to reproduce the results, supplementary materials, and demo are available at https://github.com/ChathurangiShyalika/NSF-MAP.

Problem

• Identifies anomalies in assembly pipelines for quality assurance
• Overcomes single-modality limitations in complex predictive environments
• Proposes neurosymbolic AI fusion for robust anomaly prediction

Innovation

• Neurosymbolic AI for multimodal anomaly prediction
• Time series and image-based decision-level fusion
• Transfer learning enhances fusion model performance