🤖 AI Summary
To address challenges in automatic Arabic long-text QA pair generation—including low output quality, uncontrollable difficulty levels, and the absence of standardized evaluation benchmarks—this paper proposes an end-to-end self-evolving adversarial multi-agent framework. Methodologically, it establishes a specialized large vision-language model (LVLM) collaboration system comprising a question generator, an answer ensemble, and an evaluator; integrates confidence-driven re-generation, closed-loop feedback, and automated data curation; and introduces a tunable, self-evolving difficulty mechanism. Key contributions include: (1) the first large-scale Arabic long-context evaluation benchmark, AraLongBench; (2) fully automated, human-free continual performance optimization; and (3) substantial improvements in long-context comprehension for mainstream Arabic LVLMs on AraLongBench, with superior QA pair quality and difficulty controllability compared to static pipeline approaches.
📝 Abstract
We present an end-to-end, self-evolving adversarial workflow for long-context Question-Answer (QA) Generation in Arabic. By orchestrating multiple specialized LVLMs: a question generator, an evaluator, and a swarm of answer generators, our system iteratively refines its own performance without any human intervention. Starting from raw, multi-page Arabic documents across diverse domains, the question generator produces fine-grained, context-aware queries to be tackled by the answer generator swarm, and the evaluator assesses and feeds back quality metrics. This closed-loop cycle enables continuous learning: low-confidence outputs trigger automated re-generation and model updates, progressively enhancing question difficulty and relevance. Moreover, we set the quality metrics as a tunable hyperparameter, enabling question generation at controllable and customizable difficulty levels. We release AraLongBench, a large-scale Arabic benchmark of single- and multi-page challenges spanning hundreds of pages, and demonstrate that our self-evolving workflow substantially outperform static pipelines, markedly boosting the long-context comprehension capabilities of leading Arabic Large Vision Language Models (LVLMs). Lastly, we also meticulously architect a fully automated agentic workflow for long-context Arabic document collection.