PerfCam: Digital Twinning for Production Lines Using 3D Gaussian Splatting and Vision Models

2025-04-25

AI Summary

This paper addresses key challenges in industrial digital twin deployment—particularly in pharmaceutical manufacturing—including labor-intensive 3D modeling, low-accuracy dynamic object tracking, and delayed KPI extraction. To this end, we propose PerfCam, the first open-source digital twin framework tailored for industrial settings. PerfCam innovatively integrates 3D Gaussian Splatting for lightweight, high-fidelity, semi-automatic spatial reconstruction; fuses multi-view vision with heterogeneous sensor data; and synergizes CNN-based detection/tracking with real-time spatial mapping to establish a KPI-driven dynamic twin. Evaluated on a real pharmaceutical production line, PerfCam achieves centimeter-level localization accuracy and 98.2% tracking accuracy, enabling millisecond-latency computation of critical KPIs such as Overall Equipment Effectiveness (OEE) and conveyor belt speed. Furthermore, we release the first industrial multimodal digital twin benchmark dataset to foster community advancement.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We introduce PerfCam, an open source Proof-of-Concept (PoC) digital twinning framework that combines camera and sensory data with 3D Gaussian Splatting and computer vision models for digital twinning, object tracking, and Key Performance Indicators (KPIs) extraction in industrial production lines. By utilizing 3D reconstruction and Convolutional Neural Networks (CNNs), PerfCam offers a semi-automated approach to object tracking and spatial mapping, enabling digital twins that capture real-time KPIs such as availability, performance, Overall Equipment Effectiveness (OEE), and rate of conveyor belts in the production line. We validate the effectiveness of PerfCam through a practical deployment within realistic test production lines in the pharmaceutical industry and contribute an openly published dataset to support further research and development in the field. The results demonstrate PerfCam's ability to deliver actionable insights through its precise digital twin capabilities, underscoring its value as an effective tool for developing usable digital twins in smart manufacturing environments and extracting operational analytics.

Problem

• Digital twinning for industrial production lines using 3D reconstruction.
• Semi-automated object tracking and KPI extraction in manufacturing.
• Real-time monitoring of equipment effectiveness and conveyor performance.

Innovation

• Combines 3D Gaussian Splatting with vision models
• Uses CNNs for semi-automated object tracking
• Provides real-time KPIs via digital twins