Rethinking Vector Field Learning for Generative Segmentation

2026-03-19

AI Summary

This work addresses the inherent mismatch between continuous flow matching and discrete semantic segmentation tasks, which leads to vanishing gradients, trajectory crossing, slow convergence, and blurred class boundaries. From the perspective of vector field learning, the authors propose reshaping the velocity field and introducing a distance-aware correction term to enhance inter-class separability. Additionally, they design a class encoding scheme inspired by quasi-random Kronecker sequences to enable efficient gradient propagation and pixel-level semantic alignment under end-to-end training. The proposed method significantly outperforms the original flow-matching approach and substantially narrows the performance gap between generative segmentation models and strong discriminative counterparts across multiple metrics.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Learning with Manifolds

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Taming diffusion models for generative segmentation has attracted increasing attention. While existing approaches primarily focus on architectural tweaks or training heuristics, there remains a limited understanding of the intrinsic mismatch between continuous flow matching objectives and discrete perception tasks. In this work, we revisit diffusion segmentation from the perspective of vector field learning. We identify two key limitations of the commonly used flow matching objective: gradient vanishing and trajectory traversing, which result in slow convergence and poor class separation. To tackle these issues, we propose a principled vector field reshaping strategy that augments the learned velocity field with a detached distance-aware correction term. This correction introduces both attractive and repulsive interactions, enhancing gradient magnitudes near centroids while preserving the original diffusion training framework. Furthermore, we design a computationally efficient, quasi-random category encoding scheme inspired by Kronecker sequences, which integrates seamlessly with an end-to-end pixel neural field framework for pixel-level semantic alignment. Extensive experiments consistently demonstrate significant improvements over vanilla flow matching approaches, substantially narrowing the performance gap between generative segmentation and strong discriminative specialists.

Problem

• generative segmentation
• flow matching
• vector field learning
• gradient vanishing
• trajectory traversing

Innovation

• vector field reshaping
• flow matching
• generative segmentation
• distance-aware correction
• quasi-random category encoding