Scaling Image and Video Generation via Test-Time Evolutionary Search

2025-05-23

AI Summary

Test-time scaling (TTS) for image/video generative models—particularly diffusion and flow models—lacks a general, efficient, and quality-diversity-balanced approach. Method: We propose EvoSearch, the first training-free, plug-and-play framework that systematically incorporates biological evolution principles into visual generative TTS. It formulates denoising trajectory optimization as an evolutionary search driven by stochastic differential equations, featuring adaptive population selection and noise-space mutation to dynamically allocate inference-time computation. Contribution/Results: EvoSearch is model-agnostic (diffusion/flow), modality-agnostic (image/video), and metric-agnostic, avoiding reward over-optimization. On multiple benchmarks, it achieves up to a 21% reduction in FID and over a 35% improvement in diversity, while maintaining robust gains on unseen evaluation metrics.

Technology Category

• Search and Optimization: Evolutionary Computation
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Evolutionary Learning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

As the marginal cost of scaling computation (data and parameters) during model pre-training continues to increase substantially, test-time scaling (TTS) has emerged as a promising direction for improving generative model performance by allocating additional computation at inference time. While TTS has demonstrated significant success across multiple language tasks, there remains a notable gap in understanding the test-time scaling behaviors of image and video generative models (diffusion-based or flow-based models). Although recent works have initiated exploration into inference-time strategies for vision tasks, these approaches face critical limitations: being constrained to task-specific domains, exhibiting poor scalability, or falling into reward over-optimization that sacrifices sample diversity. In this paper, we propose extbf{Evo}lutionary extbf{Search} (EvoSearch), a novel, generalist, and efficient TTS method that effectively enhances the scalability of both image and video generation across diffusion and flow models, without requiring additional training or model expansion. EvoSearch reformulates test-time scaling for diffusion and flow models as an evolutionary search problem, leveraging principles from biological evolution to efficiently explore and refine the denoising trajectory. By incorporating carefully designed selection and mutation mechanisms tailored to the stochastic differential equation denoising process, EvoSearch iteratively generates higher-quality offspring while preserving population diversity. Through extensive evaluation across both diffusion and flow architectures for image and video generation tasks, we demonstrate that our method consistently outperforms existing approaches, achieves higher diversity, and shows strong generalizability to unseen evaluation metrics. Our project is available at the website https://tinnerhrhe.github.io/evosearch.

Problem

• Understanding test-time scaling behaviors in image and video generative models
• Addressing limitations in inference-time strategies for vision tasks
• Enhancing scalability and diversity in diffusion and flow models

Innovation

• Evolutionary search optimizes denoising trajectory
• No additional training or model expansion needed
• Enhances image and video generation scalability