Self-Evolving Visual Questioner

2026-06-11

AI Summary

Existing vision-language models struggle to generate diverse, non-trivial, and visually grounded questions due to the scarcity of high-quality annotated data. This work proposes the first self-evolution framework that operates without external supervision, leveraging the model itself as both a question proposer and filter. Through an endogenous mechanism, the framework continuously generates questions that are informative, strongly visually aligned, and challenging, while jointly optimizing both question-asking and answering capabilities. Integrating self-supervised learning, in-model iterative generation and filtering, multi-dimensional question evaluation (encompassing perception, reasoning, and diversity), and a dual-modality training strategy, the approach significantly advances the quality and difficulty frontier of automatically generated questions across various backbone architectures. Under equal training budgets, it outperforms static dataset training while maintaining or even enhancing downstream question-answering performance.

Abstract

Vision-language models (VLMs) are typically trained as passive answerers, while their ability to actively ask diverse, non-trivial, visual-centric and grounded questions remains underexplored. Existing visual questioners' performance is bottlenecked by the availability of high-quality training data or the cost of curating them. We show that a VLM can continuously improve itself as a visual questioner without any external supervision. We propose a self-evolving framework that uses a VLM itself as both a proposer and a filter to produce harder, more informative, and visual-centric questions, while maintaining their exploration diversity to avoid training collapse. These questions are then used to train the VLM in both questioner and answerer modes. To evaluate the questioner, we introduce an agentic protocol that assesses questions along perception, reasoning, and diversity dimensions. Experiments across various backbone VLMs show that our method substantially enhances the quality and substantially expands the difficulty boundary of autonomous question generation. Under the same budget, our self-supervision is more effective than training on the static source data. Moreover, the self-evolving questioner remains a competitive or even better answerer.

Problem

• visual question generation
• vision-language models
• active questioning
• self-supervision
• training data bottleneck

Innovation

• self-evolving
• visual question generation
• vision-language models
• self-supervision
• agentic evaluation