JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent

2025-06-21

AI Summary

Current AI-powered photo editing tools suffer from limited adjustability and poor generalization, failing to simultaneously deliver professional-grade control and accurate interpretation of user-specific intent. This paper introduces JarvisArt—the first multimodal large language model (MLLM)-driven intelligent agent tailored for professional color grading. It employs a two-stage training paradigm: chain-of-thought supervised fine-tuning followed by color-grading-oriented Groupwise Relative Policy Optimization (GRPO-R), emulating expert artists’ decision-making to orchestrate over 200 Lightroom parameters for fine-grained global and local edits. We further design an Agent-to-Lightroom communication protocol enabling end-to-end seamless integration. Evaluated on MMArt-Bench—a real-user-constructed benchmark—JarvisArt achieves a 60% improvement in pixel-level content fidelity over GPT-4o while matching its instruction-following capability, significantly enhancing controllability and cross-scenario generalization.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Photo retouching has become integral to contemporary visual storytelling, enabling users to capture aesthetics and express creativity. While professional tools such as Adobe Lightroom offer powerful capabilities, they demand substantial expertise and manual effort. In contrast, existing AI-based solutions provide automation but often suffer from limited adjustability and poor generalization, failing to meet diverse and personalized editing needs. To bridge this gap, we introduce JarvisArt, a multi-modal large language model (MLLM)-driven agent that understands user intent, mimics the reasoning process of professional artists, and intelligently coordinates over 200 retouching tools within Lightroom. JarvisArt undergoes a two-stage training process: an initial Chain-of-Thought supervised fine-tuning to establish basic reasoning and tool-use skills, followed by Group Relative Policy Optimization for Retouching (GRPO-R) to further enhance its decision-making and tool proficiency. We also propose the Agent-to-Lightroom Protocol to facilitate seamless integration with Lightroom. To evaluate performance, we develop MMArt-Bench, a novel benchmark constructed from real-world user edits. JarvisArt demonstrates user-friendly interaction, superior generalization, and fine-grained control over both global and local adjustments, paving a new avenue for intelligent photo retouching. Notably, it outperforms GPT-4o with a 60% improvement in average pixel-level metrics on MMArt-Bench for content fidelity, while maintaining comparable instruction-following capabilities. Project Page: https://jarvisart.vercel.app/.

Problem

• Bridges gap between professional tools and AI automation
• Enhances adjustability and generalization in photo retouching
• Meets diverse and personalized editing needs intelligently

Innovation

• MLLM-driven agent for photo retouching
• Two-stage training with GRPO-R optimization
• Agent-to-Lightroom Protocol for seamless integration