🤖 AI Summary
This work addresses the complexity–performance trade-off in text encoders for visual question answering (VQA), systematically evaluating the practical efficacy of long-range dependency modeling (e.g., Transformers) versus local feature modeling (e.g., CNNs, GRUs) on the VQA-v2 benchmark. Contrary to common assumptions, it finds that complex text encoders are not universally superior; notably, lightweight local models outperform Transformers on Number/Count-type questions. To bridge this gap, the paper proposes ConvGRU—a novel architecture integrating convolutional feature enhancement into a gated recurrent unit—designed to improve counting accuracy while maintaining low computational overhead. Extensive experiments demonstrate consistent performance gains over strong baselines on VQA-v2, particularly for count-related questions. The results validate the practical utility and promise of lightweight text encoders in resource-constrained VQA scenarios, offering a compelling alternative to computationally intensive Transformer-based approaches.
📝 Abstract
Visual Question Answering (VQA) has emerged as a highly engaging field in recent years, with increasing research focused on enhancing VQA accuracy through advanced models such as Transformers. Despite this growing interest, limited work has examined the comparative effectiveness of textual encoders in VQA, particularly considering model complexity and computational efficiency. In this work, we conduct a comprehensive comparison between complex textual models that leverage long-range dependencies and simpler models focusing on local textual features within a well-established VQA framework. Our findings reveal that employing complex textual encoders is not always the optimal approach for the VQA-v2 dataset. Motivated by this insight, we propose ConvGRU, a model that incorporates convolutional layers to improve text feature representation without substantially increasing model complexity. Tested on the VQA-v2 dataset, ConvGRU demonstrates a modest yet consistent improvement over baselines for question types such as Number and Count, which highlights the potential of lightweight architectures for VQA tasks, especially when computational resources are limited.