Agentic Planning with Reasoning for Image Styling via Offline RL

2026-03-07

AI Summary

This work addresses the challenge of imprecise control in prompt-based image editing under complex style transfer tasks, where ambiguous prompts often hinder accurate manipulation. To overcome this limitation, the authors propose an agent-based planning framework grounded in tool calling, which decomposes intricate editing requests into interpretable sequences of atomic operations through chain-of-thought reasoning. The framework integrates structured context modeling and offline reinforcement learning for optimization. Key contributions include the construction of the first large-scale synthetic dataset encompassing reasoning chains, editing plans, and quality ratings; the design of a planner-oriented offline reinforcement learning strategy; and empirical validation showing that models based on Qwen3-VL 4B/8B significantly outperform baselines such as Edit-Only in composite tasks, demonstrating superior instruction-following capability and visual fidelity, as confirmed by human evaluation.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Direct prompt-based editing often fails on complex transformations because vague and subjective prompts often require nuanced understanding of what should be changed in the image. Our core intuition is that leveraging compositional image editing tools rather than direct prompting profits from structured agent-level planning with explicit reasoning, leading to better results. This structured planning framework enables efficient offline RL post-training on quality-scored trajectories to improve performance. We present a tool-based agentic RL post-training framework that addresses this through structured planning with chain-of-thought reasoning. Our key contributions include: (1) A tool-based agentic planning methodology that combines a compositional library of orthogonal primitive transformations, structured context representation, and explicit per-step reasoning to decompose complex styling into interpretable tool sequences. (2) A synthetic data generation pipeline producing three large-scale datasets (each $\sim$10K trajectories) with reasoning chains, plans, and quality scores, as no existing datasets provide such supervision. Our datasets and code are publicly available at the HuggingFace repository. (3) Offline RL training methods for learning planners with reasoning as our core algorithmic contributions, which consistently improve over the Edit-Only baseline in visual quality and instruction following. (4) Comprehensive evaluation across 4B and 8B parameter Qwen3-VL models showing that our methods outperform other baselines in the majority of compositional tasks, validated by human evaluations.

Problem

• image styling
• complex transformations
• prompt-based editing
• subjective prompts
• compositional tasks

Innovation

• Agentic Planning
• Offline Reinforcement Learning
• Chain-of-Thought Reasoning
• Compositional Image Editing
• Synthetic Data Generation