FlowSteer: Conditioning Flow Field for Consistent Image Restoration

2025-12-08

AI Summary

To address the low fidelity measurement, reliance on task-specific adapters, and poor zero-shot generalization of flow-based text-to-image (T2I) models in image restoration (IR), this paper proposes FlowSteer—a universal, adapter-free, zero-shot steering framework requiring no fine-tuning. Its core innovation is an operation-aware conditional mechanism that dynamically injects measurement residual priors along the sampling trajectory, thereby coupling the inherent generative prior of flow models with explicit data constraints to enable synergistic optimization of implicit guidance and explicit fidelity. Leveraging the noise sensitivity of flow models, FlowSteer consistently enhances reconstruction consistency and identity preservation across diverse IR tasks—including super-resolution, deblurring, denoising, and colorization—achieving state-of-the-art (SOTA) measurement fidelity in zero-shot settings.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Flow-based text-to-image (T2I) models excel at prompt-driven image generation, but falter on Image Restoration (IR), often "drifting away" from being faithful to the measurement. Prior work mitigate this drift with data-specific flows or task-specific adapters that are computationally heavy and not scalable across tasks. This raises the question "Can't we efficiently manipulate the existing generative capabilities of a flow model?" To this end, we introduce FlowSteer (FS), an operator-aware conditioning scheme that injects measurement priors along the sampling path,coupling a frozed flow's implicit guidance with explicit measurement constraints. Across super-resolution, deblurring, denoising, and colorization, FS improves measurement consistency and identity preservation in a strictly zero-shot setting-no retrained models, no adapters. We show how the nature of flow models and their sensitivities to noise inform the design of such a scheduler. FlowSteer, although simple, achieves a higher fidelity of reconstructed images, while leveraging the rich generative priors of flow models.

Problem

• Improves image restoration fidelity with flow models
• Enhances measurement consistency without retraining
• Preserves identity in zero-shot image reconstruction

Innovation

• Operator-aware conditioning scheme for flow models
• Injecting measurement priors along sampling path
• Zero-shot image restoration without retraining