Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery

2026-06-12

AI Summary

This work addresses the significant performance degradation of semantic segmentation models trained on synthetic near-infrared (NIR) images when deployed in real-world automotive scenarios, primarily caused by domain shift and scarce annotations. To mitigate this, the authors propose a generative augmentation framework comprising Target Style Adaptation (TSA) and Voronoi Style Diversification (VSD). TSA leverages low-rank fine-tuned diffusion models with structure-preserving multi-signal control to translate synthetic images into realistic NIR styles, while VSD employs geometry-guided disentanglement to decouple texture and shape inductive biases. As the first systematic study on synthetic-to-real domain adaptation for automotive NIR imagery, the proposed method reduces domain gaps by 63.6% and 28.4% in internal and external scenes, respectively, substantially enhancing the robustness and generalization of diverse segmentation models in real-world conditions.

Abstract

Semantic segmentation is a fundamental component of visual perception in modern automotive systems, enabling pixel-level scene understanding. Near-Infrared imaging (NIR) offers stable detection under difficult illumination conditions, but the development of domain-specific semantic segmentation models remains challenging due to the lack of high-quality annotated data from real-world scenarios. Synthetic datasets offer a scalable alternative, but models trained on synthetic images often suffer performance degradation when transferred to real domains. We present the first systematic study on synthetic to real domain adaptation for semantic segmentation in NIR images in the automotive domain. We propose a generative augmentation framework that transforms synthetic images into realistic NIR-style variants via our introduced target style adaptation (TSA). TSA fine-tunes a latent diffusion model via low-rank adaptation on a small curated set of real NIR images and applies it to synthetic training data using structure-preserving multi-signal conditioning. To reduce texture bias and improve segmentation robustness, we further apply a Voronoi-based style diversification strategy (VSD) that modifies the original textures while preserving scene geometry. Experiments with multiple model architectures on NIR data from vehicle interiors and street scenes show that balancing inductive bias during training leads to noticeably more robust semantic segmentation and effectively reduces the domain gap in our real-world scenarios by up to 63.6% on exterior and 28.4% on interior data. The code is available at GitHub.

Problem

• semantic segmentation
• domain adaptation
• near-infrared imagery
• synthetic-to-real transfer
• texture-shape bias

Innovation

• domain adaptation
• semantic segmentation
• near-infrared imaging
• texture-shape bias
• generative augmentation