RGB-to-IR image translation for infrared vehicle detection in unseen UAV domains

2026-09-02

AI Summary

研究通过使用生成模型（如GANs、ControlNet扩散模型等）将RGB图像转换为IR图像，以增强红外车辆检测在未知UAV领域的性能，有效缓解了IR数据稀缺问题。

Abstract

Synthetic training data is crucial for developing vision AI when real-world data is scarce, as in thermal infrared (IR) aerial vehicle detection. While abundant UAV RGB imagery motivates RGB-to-IR translation for data augmentation, unobservable thermal traits (e.g., engine heat) make learning transferable mappings challenging. This work investigates whether modern generative translators can overcome this cross-modal gap to improve infrared vehicle detection on unseen UAV target domains. Translators are trained on paired RGB-IR source datasets and applied to RGB training images from held-out target datasets to generate synthetic IR data. Evaluated methods include supervised GANs, ControlNet-based diffusion models, and foundation-model editing via LoRA. The resulting synthetic IR imagery is used to train RF-DETR vehicle detectors, which are evaluated on unseen IR target test splits across five aerial datasets, with Kust4K and VTUAV serving as target domains. Synthetic IR consistently outperforms RGB and grayscale baselines. Stable Diffusion 3.5 with ControlNet yields the best results, improving mAP from 50.8 to 60.1 on Kust4K and from 25.6 to 38.4 on VTUAV compared to models trained only on source-domain IR data. Increasing output diversity via multiple seeds (+1.1 mAP) and prompt variations (+3.3 mAP) provides additional gains on VTUAV. Although a performance gap to real target IR data remains, generative RGB-to-IR translation effectively mitigates IR data scarcity and improves cross-domain aerial vehicle detection.

Problem

• RGB-to-IR
• infrared vehicle detection
• synthetic training data
• unseen UAV domains
• thermal traits

Innovation

• RGB-to-IR translation
• generative translators
• cross-modal gap
• synthetic data augmentation
• infrared vehicle detection