EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning

2026-05-21

AI Summary

This work addresses the limitations of existing image restoration agents, which either rely on inefficient trial-and-error without training or achieve high efficiency at the cost of poor generalization when trained. The paper proposes the first training-free, self-evolving image restoration agent that systematically models experiential structures through a hierarchical experience pool. This architecture enables coarse-to-fine tool selection and guides the sequential removal of degradation while continuously updating its knowledge from accumulated interactions. By doing so, the method maintains strong compatibility with novel tools and unseen degradation types, substantially reduces trial-and-error costs, outperforms current approaches on full-reference metrics, and achieves Pareto optimality in balancing performance and efficiency.

Abstract

Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios by flexibly selecting tools and determining removal orders. However, their zero-shot planning often fails without experience, necessitating severe trial-and-error overhead to achieve satisfactory outcomes. Currently, two paradigms are employed to address this issue, yet a dilemma persists: Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradation. In contrast, training-free methods utilize explicit experience storage for compatibility but still incur trial-and-error overhead due to naive experience. To resolve the dilemma, we propose EvoIR-Agent, which first systematically formulates the experience components of a training-free image restoration agent. Subsequently, a hierarchical experience pool is constructed, which enables coarse-to-fine guidance for diverse tools and removal orders. Furthermore, a self-evolving mechanism is introduced to update the pool from scratch using accumulated records, thereby greatly improving performance and efficiency. Extensive experiments reveal that EvoIR-Agent achieves a significant lead in the full reference metrics and yields a remarkable Pareto-optimal balance between performance and efficiency compared to the state-of-the-art methods.

Problem

• image restoration
• experience-driven learning
• multimodal large language model
• trial-and-error overhead
• tool compatibility

Innovation

• self-evolving
• experience-driven learning
• hierarchical experience pool
• image restoration agent
• multimodal large language model