AndroidControl-Curated: Revealing the True Potential of GUI Agents through Benchmark Purification
Existing GUI agent evaluation benchmarks (e.g., AndroidControl) suffer from ambiguous annotations and factual inaccuracies, leading to systematic underestimation of model capabilities. To address this, we propose AndroidControl-Curated—the first high-quality benchmark jointly refined through human verification and automated curation, systematically rectifying annotation flaws in the original benchmark. We further introduce Magma-R1-3B, a lightweight yet efficient vision-language model with 3B parameters, which achieves substantial gains in instruction following and UI understanding after only 60 hours of fine-tuning on H20 GPUs. Experiments show that state-of-the-art models attain a 15-percentage-point improvement in success rate—reaching 74.8%—on the curated benchmark. Moreover, Magma-R1-3B matches the performance of Qwen3-VL-235B while using only 0.5% of its parameter count. This work establishes a more reliable evaluation standard and a practical, scalable modeling pathway for GUI agents.