SceneCraft: Interactive System for Image Editing via Scene Graph

2026-06-14

AI Summary

Existing natural language–based image editing methods struggle in complex multi-object scenes, primarily due to their reliance on users manually crafting precise textual prompts and the consequent lack of structured control. This work proposes SceneCraft, the first image editing framework that employs an editable scene graph as the human–computer interaction interface. Users can intuitively manipulate a visualized scene graph to specify spatial layouts and object relationships, which the system automatically translates into context-aware, precise editing instructions. By circumventing linguistic ambiguity and substantially reducing the burden of prompt engineering, SceneCraft enables intuitive, efficient, and accurate editing of complex relational structures. Experimental results demonstrate that SceneCraft consistently achieves higher-quality and more faithful edits across diverse tasks while offering a more intuitive user experience.

Abstract

Recent advances in generative AI have enabled natural language-driven image editing, yet existing systems often fail in complex scenes with multiple interacting objects because they rely heavily on users crafting precise text prompts. To address the absence of structured control, we propose SceneCraft, a novel interactive framework that bridges user intent and model execution by representing images as editable scene graphs. Instead of guessing text prompts through trial and error, users interact directly with a visual graph to perform complex spatial and relational operations. These graph modifications are automatically translated into precise, context-aware editing prompts, effectively eliminating linguistic ambiguity. To ensure robust and diverse results, structured prompts are dispatched to multiple state-of-the-art generative models. Evaluations across diverse editing scenarios show that SceneCraft provides a more intuitive control mechanism, significantly reducing the cognitive burden of manual prompt engineering while generating outputs that users consistently rate as higher in quality and fidelity.

Problem

• image editing
• scene graph
• natural language prompting
• structured control
• complex scenes

Innovation

• scene graph
• interactive image editing
• structured control
• prompt generation
• generative AI