🤖 AI Summary
This work addresses the common oversight of compositional aesthetics in existing image cropping methods, which often fail to produce results that are both visually appealing and interpretable. To overcome this limitation, the authors propose a novel composition-driven paradigm for explainable image cropping, reframing the task as a joint modeling of cropping, composition classification, and natural language explanation generation. They introduce COMEX, the first large-scale benchmark annotated with quadruplets comprising images, crop bounding boxes, composition categories, and human-written explanations. The dataset is constructed via image expansion and IO inversion strategies, and a two-stage training framework—combining supervised fine-tuning (SFT) and group relative policy optimization (GRPO)—is employed to optimize multi-task outputs. Experiments demonstrate that the proposed approach significantly outperforms existing methods in cropping quality, composition prediction accuracy, and explanation faithfulness, while also providing a unified evaluation platform for 15 vision-language models.
📝 Abstract
Explainable aesthetic image cropping requires not only localizing a visually pleasing crop but also explaining why it is preferred. Existing crop-and-explain methods largely treat explanation as post-hoc text generation and overlook composition, a key aesthetic factor that links crop decisions with interpretable reasoning. In this paper, we reformulate explainable aesthetic image cropping as a structured crop-composition-explanation problem. To support this setting, we introduce COMEX, a new benchmark built through image expansion and an IO-reversal pipeline. COMEX contains 33,161 quadruples, each consisting of an expanded image, a crop box, a composition category, and a composition-grounded explanation, enabling joint learning of crop localization, composition understanding, and explanation generation. We further propose a two-stage SFT+GRPO framework, where supervised fine-tuning establishes the structured output protocol and basic cropping ability, and GRPO further improves crop quality, composition prediction, and explanation faithfulness. We benchmark 15 large vision-language models and existing cropping methods on COMEX, establishing a comprehensive testbed for composition-grounded explainable aesthetic cropping. Experiments on both COMEX and prior benchmarks demonstrate the effectiveness and transferability of our framework, with strong performance across evaluation metrics.