PhotoAgent: Agentic Photo Editing with Exploratory Visual Aesthetic Planning

2026-02-26

AI Summary

This work addresses the challenge of autonomous image editing, which existing methods struggle to achieve due to their reliance on detailed user instructions. The authors formulate autonomous image editing as a long-horizon decision-making problem and propose the first intelligent editing system that integrates explicit aesthetic planning, exploratory multi-step action tree search, and closed-loop visual feedback. Central to this framework is a learned aesthetic reward model that guides the editing process. Key contributions include a novel closed-loop architecture equipped with memory and feedback mechanisms, as well as the release of UGC-Edit—the first benchmark for aesthetic evaluation in real-world editing scenarios. Evaluated on a test set of 1,017 photographs, the proposed method significantly outperforms existing baselines in both instruction adherence and visual quality.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

With the recent fast development of generative models, instruction-based image editing has shown great potential in generating high-quality images. However, the quality of editing highly depends on carefully designed instructions, placing the burden of task decomposition and sequencing entirely on the user. To achieve autonomous image editing, we present PhotoAgent, a system that advances image editing through explicit aesthetic planning. Specifically, PhotoAgent formulates autonomous image editing as a long-horizon decision-making problem. It reasons over user aesthetic intent, plans multi-step editing actions via tree search, and iteratively refines results through closed-loop execution with memory and visual feedback, without requiring step-by-step user prompts. To support reliable evaluation in real-world scenarios, we introduce UGC-Edit, an aesthetic evaluation benchmark consisting of 7,000 photos and a learned aesthetic reward model. We also construct a test set containing 1,017 photos to systematically assess autonomous photo editing performance. Extensive experiments demonstrate that PhotoAgent consistently improves both instruction adherence and visual quality compared with baseline methods. The project page is https://github.com/mdyao/PhotoAgent.

Problem

• autonomous image editing
• aesthetic planning
• instruction-based editing
• visual feedback
• long-horizon decision-making

Innovation

• autonomous image editing
• aesthetic planning
• tree search
• closed-loop refinement
• generative models