GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization

2026-01-26

AI Summary

This work addresses the inefficiency of existing slider-based manual tuning methods for fusing numerous image generation adapters, which struggle to effectively explore high-dimensional weight spaces. To overcome this limitation, we propose an interactive model fusion framework that employs a two-stage preference-based Bayesian optimization (PBO) backend. By incorporating adapter weight constraints and leveraging the sparsity of user feedback, our approach significantly improves sampling efficiency and convergence speed in high-dimensional weight spaces. Users can efficiently explore diverse visual outputs through preference feedback, and extensive experiments—both with simulated and real users—demonstrate that our method consistently outperforms conventional Bayesian optimization and line search baselines, achieving higher success rates and faster convergence.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Fine-tuning-based adaptation is widely used to customize diffusion-based image generation, leading to large collections of community-created adapters that capture diverse subjects and styles. Adapters derived from the same base model can be merged with weights, enabling the synthesis of new visual results within a vast and continuous design space. To explore this space, current workflows rely on manual slider-based tuning, an approach that scales poorly and makes weight selection difficult, even when the candidate set is limited to 20-30 adapters. We propose GimmBO to support interactive exploration of adapter merging for image generation through Preferential Bayesian Optimization (PBO). Motivated by observations from real-world usage, including sparsity and constrained weight ranges, we introduce a two-stage BO backend that improves sampling efficiency and convergence in high-dimensional spaces. We evaluate our approach with simulated users and a user study, demonstrating improved convergence, high success rates, and consistent gains over BO and line-search baselines, and further show the flexibility of the framework through several extensions.

Problem

• adapter merging
• image generation
• weight selection
• design space exploration
• interactive optimization

Innovation

• Bayesian Optimization
• Adapter Merging
• Diffusion Models
• Interactive Image Generation
• Preferential Optimization