OmniFuser: Adaptive Multimodal Fusion for Service-Oriented Predictive Maintenance

2025-11-03

AI Summary

To address untimely tool condition prediction—leading to quality degradation and unplanned downtime—in intelligent manufacturing, this paper proposes a vision-force multimodal fusion framework tailored for milling processes. Methodologically, we design a contamination-free cross-modal disentangled fusion mechanism that separates shared representations from modality-specific features; further, we introduce a recursive refinement pathway and a residual information anchoring strategy to enhance fusion robustness, and employ a parallel feature extraction network for efficient dynamic integration of heterogeneous signals. Evaluated on a real-world milling dataset, our approach significantly outperforms state-of-the-art methods, achieving high-accuracy three-class classification (sharp/used/blunt) of tool states and accurate multi-step cutting force prediction. The framework delivers a reliable, reusable technical foundation for industrial intelligent maintenance systems.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Accurate and timely prediction of tool conditions is critical for intelligent manufacturing systems, where unplanned tool failures can lead to quality degradation and production downtime. In modern industrial environments, predictive maintenance is increasingly implemented as an intelligent service that integrates sensing, analysis, and decision support across production processes. To meet the demand for reliable and service-oriented operation, we present OmniFuser, a multimodal learning framework for predictive maintenance of milling tools that leverages both visual and sensor data. It performs parallel feature extraction from high-resolution tool images and cutting-force signals, capturing complementary spatiotemporal patterns across modalities. To effectively integrate heterogeneous features, OmniFuser employs a contamination-free cross-modal fusion mechanism that disentangles shared and modality-specific components, allowing for efficient cross-modal interaction. Furthermore, a recursive refinement pathway functions as an anchor mechanism, consistently retaining residual information to stabilize fusion dynamics. The learned representations can be encapsulated as reusable maintenance service modules, supporting both tool-state classification (e.g., Sharp, Used, Dulled) and multi-step force signal forecasting. Experiments on real-world milling datasets demonstrate that OmniFuser consistently outperforms state-of-the-art baselines, providing a dependable foundation for building intelligent industrial maintenance services.

Problem

• Predicts milling tool conditions using multimodal visual and sensor data
• Integrates heterogeneous features through contamination-free cross-modal fusion
• Enables reliable tool-state classification and force signal forecasting

Innovation

• Fuses visual and sensor data for predictive maintenance
• Employs contamination-free cross-modal fusion mechanism
• Uses recursive refinement to stabilize fusion dynamics