Enabling 24-hour Agricultural Robotics: Unsupervised Day-to-Night Cross-Modal Image Translation for Nighttime Visual Navigation

2026-07-13

AI Summary

This study addresses the challenge that existing agricultural robot vision systems rely on daytime imagery and struggle to operate at night due to the scarcity of annotated nighttime data. To bridge this gap, the authors propose an unsupervised cross-modal image translation method that converts RGB daytime images into near-infrared (NIR) nighttime-like images, enabling the transfer of daytime semantic labels to nighttime perception. The approach innovatively leverages a pre-trained CLIP model to preserve semantic consistency and introduces a visibility mask to model the effective illumination range of NIR lighting. Based on this framework, the authors construct AgriNight, the first benchmark dataset for nighttime agricultural visual navigation. Experiments demonstrate that the proposed method significantly enhances nighttime image quality and downstream semantic segmentation performance, successfully enabling real-world autonomous robot navigation in nighttime field conditions.

Abstract

While visual navigation has been extensively studied in agricultural robotics, most existing systems assume daytime conditions. In fact, deploying autonomous robots at night offers significant advantages, including 24-hour crop and soil monitoring, fruit harvesting, and nocturnal pest detection. Modern vision-based systems, however, rely heavily on large-scale well-annotated image datasets, which remains challenging to obtain for nighttime operation scenarios. To address this, we propose an unsupervised image translation framework that converts daytime plant-row RGB images into near-infrared (NIR) nighttime counterparts without requiring pixel-to-pixel supervision. This enables the direct reuse of daytime semantic labels for training nighttime perception models. In particular, by incorporating a pre-trained Contrastive Language-Image Pre-training (CLIP) model, the proposed framework is designed to preserve semantic consistency during day-to-night translation. Additionally, a visibility mask is introduced to account for the limited effective range of NIR illumination in nighttime scenes. We conduct comparative evaluations with state-of-the-art image translation baselines and demonstrate higher image qualities, as supported by improved performance in downstream semantic segmentation for nighttime visual navigation. For evaluation, we utilize AgriNight--a novel dataset comprising 428 daytime and 549 nighttime images collected using night-vision-equipped mobile robots in agricultural fields and manually annotated with pixel-wise semantic labels--and introduce it as the first benchmark for nighttime agricultural visual navigation. We also perform real-time autonomous navigation experiments with a physical robot operating at night. The data and code are available at: https://github.com/mamorobel/AgriNight.

Problem

• agricultural robotics
• nighttime visual navigation
• image translation
• semantic segmentation
• unsupervised learning

Innovation

• unsupervised image translation
• nighttime visual navigation
• cross-modal learning
• CLIP-based semantic consistency
• agricultural robotics