One Image is All You Need: Agentic One-Shot Image Generation via Text-Based World Models for Long-Tail Spatial Perception

2026-06-18

AI Summary

This work addresses the degradation of visual perception performance in safety-critical scenarios under long-tailed data distributions, where real-world training samples are scarce. To tackle this challenge, the authors propose WMGen-v1, a novel framework that leverages only a single reference image to generate diverse, spatially consistent synthetic data. WMGen-v1 uniquely integrates a text-based world model with an agent mechanism, constructing structured scene representations grounded in physical plausibility and commonsense constraints to guide a diffusion model. The approach forms a multi-stage pipeline combining large vision-language models, large language models, and diffusion models to jointly enable reasoning and generation. Experiments demonstrate that detectors trained exclusively on the synthesized data significantly outperform existing methods across multiple industrial and public benchmarks, achieving performance close to that of models trained on real data, thereby effectively mitigating the long-tail data scarcity problem.

Abstract

Reliable spatial decision automation, such as autonomous driving and maritime surveillance, critically depends on robust visual perception. However, real-world spatiotemporal data exhibits severe heterogeneity, often manifesting as extreme long-tail distributions for safety-critical scenarios. This data scarcity induces dataset shift that degrades detection performance and pose safety risks. While synthetic data generation offers a potential solution, existing generative approaches, such as diffusion models and Generative Adversarial Networks (GANs), often lack explicit spatial grounding and structural constraints, resulting in spatial and physical inconsistencies in generated scenes. To address these challenges, we introduce WMGen-v1, an agentic text-based world model framework for long-tail spatial data generation. WMGen-v1 employs a Large Vision-Language Model (LVLM) to construct a structured scene representation from a single reference image, while a Large Language Model (LLM) performs guidance-based scene expansion under physical plausibility and commonsense constraints. Subsequently, conditioned on the structured semantic representations produced by this reasoning process, a diffusion model generates diverse and physically grounded long-tail training data. Experiments on internal industrial datasets, ROADWork, and LaRS benchmarks demonstrate that WMGen-v1 outperforms baseline approaches. Notably, detectors trained solely on WMGen-v1 synthetic data approach real-only performance on aggregate dataset-level metrics, highlighting its potential to alleviate long-tail data scarcity for downstream spatial perception.

Problem

• long-tail distribution
• spatial perception
• synthetic data generation
• dataset shift
• physical consistency

Innovation

• world model
• one-shot image generation
• long-tail spatial perception
• vision-language model
• physically grounded synthesis