Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation

2026-02-17

AI Summary

This work addresses the challenge of semantic hallucination in unpaired day-to-night image translation, where large appearance discrepancies and the absence of pixel-level supervision often lead to distorted representations of critical objects such as traffic signs and vehicles, thereby degrading downstream task performance. To mitigate this issue, the authors propose a novel framework that employs a dual-headed discriminator to identify hallucinated content in the background and leverages target-domain annotations to construct class-specific prototypes as semantic anchors. These prototypes repel hallucinatory features in the feature space, preserving semantic consistency. The method uniquely integrates class prototypes, a dual-headed discriminator, and a Schrödinger bridge model within an iterative optimization scheme. Evaluated on BDD100K, it achieves a 15.5% improvement in mean average precision (mAP) for day-to-night domain adaptation, with particularly challenging classes like traffic lights showing gains up to 31.7%, substantially outperforming existing approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Day-to-night unpaired image translation is important to downstream tasks but remains challenging due to large appearance shifts and the lack of direct pixel-level supervision. Existing methods often introduce semantic hallucinations, where objects from target classes such as traffic signs and vehicles, as well as man-made light effects, are incorrectly synthesized. These hallucinations significantly degrade downstream performance. We propose a novel framework that detects and suppresses hallucinations of target-class features during unpaired translation. To detect hallucination, we design a dual-head discriminator that additionally performs semantic segmentation to identify hallucinated content in background regions. To suppress these hallucinations, we introduce class-specific prototypes, constructed by aggregating features of annotated target-domain objects, which act as semantic anchors for each class. Built upon a Schrodinger Bridge-based translation model, our framework performs iterative refinement, where detected hallucination features are explicitly pushed away from class prototypes in feature space, thus preserving object semantics across the translation trajectory.Experiments show that our method outperforms existing approaches both qualitatively and quantitatively. On the BDD100K dataset, it improves mAP by 15.5% for day-to-night domain adaptation, with a notable 31.7% gain for classes such as traffic lights that are prone to hallucinations.

Problem

• unpaired image translation
• semantic hallucination
• day-to-night translation
• object hallucination
• domain adaptation

Innovation

• hallucination suppression
• unpaired image translation
• class-specific prototypes
• dual-head discriminator
• Schrödinger Bridge