VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance

2026-07-16

AI Summary

This work addresses the lack of systematic evaluation of vision-language models in real-world assistive scenarios for people with visual impairments. To bridge this gap, the authors introduce VIABench, the first first-person video benchmark collected by visually impaired users, encompassing three core tasks: proactive alerting, visual question answering, and visually guided interaction, with support for both online and offline evaluation. Emphasizing forward-looking event prediction and real-time responsiveness, VIABench establishes the first comprehensive evaluation framework grounded in authentic user needs. Experimental results reveal that current multimodal large language models exhibit limited performance on critical tasks such as proactive alerting, underscoring the necessity for purpose-built models and providing a reliable benchmark for future research in assistive vision-language systems.

Abstract

Visually impaired individuals (VIIs) encounter significant daily challenges due to limited access to visual information. Although Multimodal Large Language Models (MLLMs) have achieved impressive results on general vision and language tasks, their practical utility in real-world blind assistance still remains largely underexplored. To fill this gap, we introduce VIABench, a comprehensive video benchmark specifically designed to evaluate MLLMs in Visually Impaired Assistance scenarios using first-person videos recorded or shared by VIIs themselves. VIABench defines three core tasks, each targeting a distinct requirement in visual assistance. Proactive Reminder: Assesses the model's ability to interpret ongoing video content while proactively anticipating and verbally describing upcoming navigation-critical events; Visual Question Answering (VQA): Evaluates the model's capacity to answer user-posed questions about the environment or objects within the video; Vision-Guided Interaction: Tests context-aware reasoning to accomplish intentional interactions between user and environment. To ensure a robust and fair evaluation, we propose a rigorous benchmarking pipeline that supports both online (real-time) and offline settings. Our experiments demonstrate that current MLLMs still struggle to deliver comprehensive support for VIIs, especially in the Proactive Reminder task, which demands accurate anticipation and real-time responsiveness. We hope VIABench will drive future research toward developing customized MLLMs for real-world assistance, ultimately improving navigation and interaction experiences for visually impaired individuals. Code and data will be released at https://github.com/MCG-NJU/VIABench.

Problem

• Visually Impaired Assistance
• Multimodal Large Language Models
• Video Benchmark
• First-person Vision
• Real-world Evaluation

Innovation

• VIABench
• Multimodal Large Language Models
• Proactive Reminder
• First-person Video
• Visual Impairment Assistance