Thinking Before Retrieving: Robust Zero-Shot Composed Image Retrieval via Strategic Planning and Self-Criticism

2026-06-30

AI Summary

This work addresses the challenge in zero-shot compositional image retrieval where single-pass generation of query text often suffers from semantic distortion or missing attributes, degrading retrieval accuracy. To mitigate this, the authors propose PEC-CIR, a novel framework that introduces, for the first time, a three-stage reasoning architecture—Plan, Execute, Critique. The planner explicitly extracts constraints from the reference image and textual instruction; the executor generates candidate descriptions; and the critic evaluates their adherence to the extracted constraints. Operating within a frozen vision-language embedding space, PEC-CIR leverages large language models to perform multi-step reasoning and self-critique without any additional training. This decouples constraint parsing, description generation, and quality assessment, effectively curbing error propagation and substantially improving both retrieval accuracy and robustness.

Abstract

Composed image retrieval requires identifying a target image from a gallery by integrating a reference image with a textual modification instruction. In a training-free zero-shot setting, this task relies on constructing a retrieval-oriented textual query within a frozen vision--language embedding space at inference time. Existing approaches predominantly rely on a single-pass generation strategy that fuses the reference context and modification text into a unified description. This strategy makes it difficult to detect or correct semantic distortions and omissions during generation. Consequently, the preservation of reference attributes and the integration of textual requirements interfere with each other, which degrades retrieval precision. To address these challenges, we introduce PEC-CIR, a training-free framework that structures query construction as a multi-stage reasoning pipeline. The framework operates through a Planner--Executor--Critic architecture where the Planner extracts explicit constraints, the Executor generates multiple candidate target descriptions, and the Critic evaluates these candidates based on constraint compliance. By reframing query construction as a staged inference process instead of a single-pass output, PEC-CIR reduces the propagation of generative errors by explicitly evaluating candidate queries before retrieval, thereby improving retrieval stability.

Problem

• composed image retrieval
• zero-shot
• semantic distortion
• query construction
• vision-language embedding

Innovation

• zero-shot composed image retrieval
• strategic planning
• self-criticism
• multi-stage reasoning
• query construction