NIFTY: a Non-Local Image Flow Matching for Texture Synthesis

2025-09-26

AI Summary

This paper addresses exemplar-based texture synthesis, proposing a training-free, non-parametric method. The core innovation integrates flow matching—a mechanism from diffusion models—with classical non-local means (NLM) block matching: pixel-level image flow fields are constructed via block matching, enabling flow matching to replace conventional iterative optimization; additionally, a lightweight CNN-guided diffusion prior is introduced to enhance structural coherence and fine-detail fidelity. By circumventing neural network training and initialization sensitivity, the approach significantly mitigates artifacts and blurring. Extensive evaluation across diverse texture synthesis tasks demonstrates superior visual quality and quantitative performance over representative baselines—particularly in reconstructing complex structures and high-frequency details.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper addresses the problem of exemplar-based texture synthesis. We introduce NIFTY, a hybrid framework that combines recent insights on diffusion models trained with convolutional neural networks, and classical patch-based texture optimization techniques. NIFTY is a non-parametric flow-matching model built on non-local patch matching, which avoids the need for neural network training while alleviating common shortcomings of patch-based methods, such as poor initialization or visual artifacts. Experimental results demonstrate the effectiveness of the proposed approach compared to representative methods from the literature. Code is available at https://github.com/PierrickCh/Nifty.git

Problem

• Exemplar-based texture synthesis using hybrid framework
• Combines diffusion models with patch-based optimization
• Avoids neural training while reducing patch artifacts

Innovation

• Non-parametric flow-matching model using non-local patches
• Combines diffusion models with patch-based optimization
• Avoids neural network training while reducing artifacts