Applying Deep Learning for cockpit segmentation in the context of mixed reality
This study addresses the demand for precise foreground-background segmentation in mixed reality (MR) applications involving heavy machinery cockpits, where seamless integration of virtual and real elements is critical. For the first time, U-Net and DeepLabV3+ are applied to semantic segmentation of first-person cockpit imagery from a mining truck simulator. Trained on a dataset collected from real-world scenarios, both models achieve approximately 90% segmentation accuracy while maintaining real-time performance. Experimental results demonstrate that the selected architectures effectively and accurately delineate cockpit foregrounds from backgrounds, significantly enhancing immersion and compositing quality in MR environments. This work thus provides robust foundational visual support for MR systems deployed with heavy equipment.