Zero-Shot Visual Concept Blending Without Text Guidance

2025-03-27

AI Summary

This work addresses the problem of fine-grained, text-free visual feature selection and fusion from multiple reference images. We propose the first zero-shot, multi-image-driven visual concept disentanglement and fusion framework: building upon IP-Adapter, it constructs a locally disentangled CLIP embedding space; through cross-reference feature contrastive modeling, it automatically separates shared and image-specific visual attributes—including texture, shape, style, and abstract concepts (e.g., “dynamism” or “aerodynamic lines”)—enabling training-free, text-free, attribute-selective transfer across images. Our method achieves high-fidelity fusion in style transfer, morphological deformation, and concept composition tasks. A user study confirms 92% accuracy in target feature identification, demonstrating effective semantic controllability and perceptual fidelity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

We propose a novel, zero-shot image generation technique called"Visual Concept Blending"that provides fine-grained control over which features from multiple reference images are transferred to a source image. If only a single reference image is available, it is difficult to isolate which specific elements should be transferred. However, using multiple reference images, the proposed approach distinguishes between common and unique features by selectively incorporating them into a generated output. By operating within a partially disentangled Contrastive Language-Image Pre-training (CLIP) embedding space (from IP-Adapter), our method enables the flexible transfer of texture, shape, motion, style, and more abstract conceptual transformations without requiring additional training or text prompts. We demonstrate its effectiveness across a diverse range of tasks, including style transfer, form metamorphosis, and conceptual transformations, showing how subtle or abstract attributes (e.g., brushstroke style, aerodynamic lines, and dynamism) can be seamlessly combined into a new image. In a user study, participants accurately recognized which features were intended to be transferred. Its simplicity, flexibility, and high-level control make Visual Concept Blending valuable for creative fields such as art, design, and content creation, where combining specific visual qualities from multiple inspirations is crucial.

Problem

• Blending visual features from multiple reference images without text guidance
• Isolating and transferring specific elements using multiple references
• Enabling flexible control over texture, shape, and abstract transformations

Innovation

• Zero-shot image generation without text guidance
• Selective feature transfer from multiple references
• CLIP embedding space for flexible visual transformations