AI-based worker guidance in assembly and disassembly operations using multimodal ego/exo-centric data capture and structured task knowledge

2026-08-23

AI Summary

本文通过使用第一人称和第三人称视角的数据采集与结构化任务知识，解决了装配和拆卸过程中专家知识难以记录、复用及转移的问题。

Abstract

Assembly and disassembly processes rely on expert knowledge that is difficult to document, reuse, and transfer. This paper presents a data-centric approach for extracting structured task knowledge from expert demonstrations using egocentric and exocentric recordings. Temporal and multimodal information from video and narration is jointly encoded to derive structured task representations that enable procedural documentation and context-aware worker guidance. The approach is evaluated on a real-world disassembly case study, demonstrating that video-based representations capture procedural structure and execution context beyond static image-based methods. The results highlight the potential of egocentric video understanding for repair, training, and circular manufacturing applications. Project website: https://indego-assistant.github.io/

Problem

• assembly and disassembly
• expert knowledge
• structured task knowledge
• multimodal data
• egocentric and exocentric recordings

Innovation

• multimodal data
• structured task knowledge
• egocentric and exocentric recordings