RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models

2026-08-14

AI Summary

This study addresses the challenge of identifying editable semantics in text-to-image models, which traditionally relies on labor-intensive manual trial-and-error. We propose a training-free, model-agnostic framework that formulates semantic discovery as a subset selection problem for the first time. By integrating multimodal vision-language models with submodular optimization algorithms, this approach efficiently identifies semantic concepts that are relevant, editable, and diverse. Experimental results demonstrate that our method accurately discovers a broad spectrum of editable semantics across multiple domains, significantly outperforming existing baselines. Consequently, this work substantially enhances image editing efficiency while reducing the reliance on costly human intervention, offering a scalable solution for semantic manipulation in generative models.

Abstract

Recent advances in text-to-image (T2I) models have revolutionized the field of image generation and editing. However, identifying semantics that a T2I model can successfully edit in an image continues to be a challenging task. Most existing approaches require users to manually specify semantics to modify a particular image, a time-consuming process that often involves extensive trial and error. In this paper, we present RankT2I, a novel, training-free, and model-agnostic framework that automates the discovery of editable semantics in diffusion and FLUX-based models. Given a visual domain, we first utilize a multimodal vision-language model to gather a broad set of candidate semantics. We then frame semantic discovery as a set selection problem and use a submodular objective to identify semantics that are relevant, editable, and diverse. Our method helps users efficiently identify a wide range of semantics for text-to-image editing models across several domains while outperforming existing methods.

Problem

• Text-to-Image Editing
• Editable Semantics Discovery
• Semantic Identification
• Manual Specification

Innovation

• Submodular Optimization
• Training-free Framework
• Editable Semantics Discovery
• Model-agnostic
• Set Selection Problem