Generalizable Image Repair for Robust Visual Autonomous Racing

2025-03-07

AI Summary

To address image distribution shifts in vision-based autonomous racing caused by sensor noise, adverse weather, and dynamic illumination, this paper proposes a lightweight, real-time image restoration module that recovers corrupted observations prior to control decision-making, thereby enhancing perceptual robustness and control reliability. Methodologically, we introduce a novel control-oriented joint loss function that jointly optimizes perceptual fidelity and downstream control performance. Our architecture synergistically integrates CycleGAN—enabling cross-domain generalization without paired data—and pix2pix—ensuring high-fidelity reconstruction with paired data—yielding strong generalization to unseen corruption types. Evaluated in a simulated racing environment, our approach significantly outperforms baseline methods: controller success rates improve markedly, while inference latency remains low and computational overhead stays within practical limits.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Vision-based autonomous racing relies on accurate perception for robust control. However, image distribution changes caused by sensor noise, adverse weather, and dynamic lighting can degrade perception, leading to suboptimal control decisions. Existing approaches, including domain adaptation and adversarial training, improve robustness but struggle to generalize to unseen corruptions while introducing computational overhead. To address this challenge, we propose a real-time image repair module that restores corrupted images before they are used by the controller. Our method leverages generative adversarial models, specifically CycleGAN and pix2pix, for image repair. CycleGAN enables unpaired image-to-image translation to adapt to novel corruptions, while pix2pix exploits paired image data when available to improve the quality. To ensure alignment with control performance, we introduce a control-focused loss function that prioritizes perceptual consistency in repaired images. We evaluated our method in a simulated autonomous racing environment with various visual corruptions. The results show that our approach significantly improves performance compared to baselines, mitigating distribution shift and enhancing controller reliability.

Problem

• Address image corruption in autonomous racing perception
• Improve robustness against sensor noise and adverse conditions
• Enhance controller reliability with real-time image repair

Innovation

• Real-time image repair using generative adversarial models
• CycleGAN and pix2pix for image-to-image translation
• Control-focused loss function for perceptual consistency