🤖 AI Summary
This study addresses the demand for precise foreground-background segmentation in mixed reality (MR) applications involving heavy machinery cockpits, where seamless integration of virtual and real elements is critical. For the first time, U-Net and DeepLabV3+ are applied to semantic segmentation of first-person cockpit imagery from a mining truck simulator. Trained on a dataset collected from real-world scenarios, both models achieve approximately 90% segmentation accuracy while maintaining real-time performance. Experimental results demonstrate that the selected architectures effectively and accurately delineate cockpit foregrounds from backgrounds, significantly enhancing immersion and compositing quality in MR environments. This work thus provides robust foundational visual support for MR systems deployed with heavy equipment.
📝 Abstract
Computer vision is an area that has been growing continuously. With the advance of technologies with a first-person view, new development opportunities have emerged inside the area. Mixed reality promotes virtual environments with objects from the physical world shown in real time. For that, it's necessary to be concerned with the immersion of the user in this simulated environment, increasingly seeking to bring it closer to a possible desired reality. This paper proposes the development of image processing in order to perform the segmentation of images to identify what is foreground and background in order to facilitate the union of virtual and real images. Thus, the present work obtain real images of the user using the off-highway truck simulator CAT793F, through a camera, to be able to perform the segmentation of such images with artificial intelligence techniques.The convolutional neural network architectures "U-net" and "DeepLabV3+" are applied to perform image segmentation. As a result, metrics with around 90% accuracy were presented and and the best model was determined.