Enhancing Human-Robot Collaboration: A Sim2Real Domain Adaptation Algorithm for Point Cloud Segmentation in Industrial Environments

2025-06-11

AI Summary

To address the scarcity of annotated real-world point cloud semantic segmentation data, weak model generalization, and insufficient Sim2Real transfer robustness in industrial human-robot collaboration (HRC), this paper proposes FUSION—a dual-stream network tailored for HRC. It introduces a novel architecture integrating dynamic graph convolutional neural networks (DGCNN) and residual CNNs in parallel streams, coupled with feature-space alignment to enable end-to-end domain adaptation from simulation to reality. Crucially, the method operates without requiring real-world ground-truth labels. Evaluated on real HRC production lines and corresponding synthetic data, it achieves 97.76% mean Intersection-over-Union (mIoU), significantly outperforming state-of-the-art approaches. The framework delivers high accuracy, strong robustness against domain shift, and practical deployability—thereby effectively supporting safe and efficient human-robot co-working.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Visual Reasoning & Symbolic Representations
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

The robust interpretation of 3D environments is crucial for human-robot collaboration (HRC) applications, where safety and operational efficiency are paramount. Semantic segmentation plays a key role in this context by enabling a precise and detailed understanding of the environment. Considering the intense data hunger for real-world industrial annotated data essential for effective semantic segmentation, this paper introduces a pioneering approach in the Sim2Real domain adaptation for semantic segmentation of 3D point cloud data, specifically tailored for HRC. Our focus is on developing a network that robustly transitions from simulated environments to real-world applications, thereby enhancing its practical utility and impact on a safe HRC. In this work, we propose a dual-stream network architecture (FUSION) combining Dynamic Graph Convolutional Neural Networks (DGCNN) and Convolutional Neural Networks (CNN) augmented with residual layers as a Sim2Real domain adaptation algorithm for an industrial environment. The proposed model was evaluated on real-world HRC setups and simulation industrial point clouds, it showed increased state-of-the-art performance, achieving a segmentation accuracy of 97.76%, and superior robustness compared to existing methods.

Problem

• Enhancing human-robot collaboration via Sim2Real domain adaptation
• Improving 3D point cloud segmentation for industrial environments
• Bridging simulation-to-reality gap for safer HRC applications

Innovation

• Sim2Real domain adaptation for point clouds
• Dual-stream DGCNN and CNN architecture
• Enhanced segmentation accuracy and robustness