Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation

2026-02-03

AI Summary

This work addresses the inefficiency of existing inference-time optimization methods in high-dimensional noise spaces, where most perturbation directions have negligible impact on generation outcomes. We propose a plug-and-play low-frequency subspace evolutionary search framework that efficiently optimizes initial noise without gradients to align image generation with downstream reward objectives. Inspired by the spectral bias observed in generative processes, our approach focuses on the low-frequency subspace to which outputs are most sensitive, and we develop a perturbation propagation dynamics theory to justify its efficacy. Experiments demonstrate that, under identical computational budgets, our method significantly outperforms current baselines, advancing the Pareto frontier between generation quality and computational cost.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Inference-time scaling offers a versatile paradigm for aligning visual generative models with downstream objectives without parameter updates. However, existing approaches that optimize the high-dimensional initial noise suffer from severe inefficiency, as many search directions exert negligible influence on the final generation. We show that this inefficiency is closely related to a spectral bias in generative dynamics: model sensitivity to initial perturbations diminishes rapidly as frequency increases. Building on this insight, we propose Spectral Evolution Search (SES), a plug-and-play framework for initial noise optimization that executes gradient-free evolutionary search within a low-frequency subspace. Theoretically, we derive the Spectral Scaling Prediction from perturbation propagation dynamics, which explains the systematic differences in the impact of perturbations across frequencies. Extensive experiments demonstrate that SES significantly advances the Pareto frontier of generation quality versus computational cost, consistently outperforming strong baselines under equivalent budgets.

Problem

• inference-time scaling
• initial noise optimization
• spectral bias
• reward-aligned generation
• computational efficiency

Innovation

• Spectral Evolution Search
• inference-time scaling
• spectral bias
• low-frequency subspace
• gradient-free optimization