Residual-Space Evolutionary Optimization via Flow-based Generative Models

2026-06-18

AI Summary

This work addresses the challenge of optimizing non-differentiable or black-box objectives in flow-based generative editing by proposing a model-agnostic evolutionary optimization framework in residual space. The method introduces evolutionary algorithms into the residual space of flow-based generative models for the first time, leveraging self-mating for local refinement and crossover for global exploration. This gradient-free approach effectively decouples conditional control from instance-specific residuals. Experiments demonstrate that the framework achieves a favorable balance among target alignment, instance fidelity, and diversity on both MorphoMNIST and crystal datasets, showcasing its effectiveness and versatility for efficient data editing in image synthesis and real-world scientific applications.

Abstract

Data editing with generative methods typically requires differentiable objectives and gradient-based search. However, these assumptions break down in flow-based settings, where edits are performed through forward and backward integration and often involve non-differentiable or black-box objectives. We introduce residual-space evolutionary optimization, a model-agnostic framework that addresses this gap by combining flow-based generative editing with evolutionary algorithms. Building on the observation that conditional flow matching (CFM) can disentangle condition-controlled factors from instance-specific residuals, our framework directly operates in residual space and separates two complementary search regimes: self-pollination performs local exploitation through feature-preserving residual refinement, and cross-pollination promotes broader exploration by recombining residuals across heterogeneous samples. As a proof of concept, we validate on MorphoMNIST, a benchmark dataset for counterfactual generation, and on crystal data, demonstrating that this exploration--exploitation decomposition provides a useful mechanism for balancing target alignment, instance preservation, and diversity, and extends beyond images to real-world scientific domains.

Problem

• flow-based generative models
• data editing
• non-differentiable objectives
• black-box optimization
• residual space

Innovation

• residual-space optimization
• flow-based generative models
• evolutionary algorithms
• conditional flow matching
• counterfactual generation