MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places

2026-08-28

AI Summary

MAP基准用于评估多模态AI系统在满足现实世界地点访问需求时的表现，通过验证或推荐符合无障碍要求的兴趣点来解决信息准确性和视觉证据检索问题。

Abstract

We introduce MAP, the first benchmark to evaluate multimodal AI systems as assistants for users with accessibility requirements when planning visits to places in the real world. In our evaluation, systems are presented with requests to verify or recommend a point of interest meeting an accessibility requirement. MAP contains two novel assessments: Claim verification for accessibility planning assesses if information on places and stated accessibility features is supported and identifies places that satisfy requested accessibility features. Visual evidence retrieval for accessibility planning checks if a multimodal AI system can select visual evidence for the requested place and accessibility feature. Our methodology supports comparison of AI systems in a setting where place information and accessibility information can change over time by evaluating systems and refreshing ground truth data at scheduled times. The benchmark is based on automatic rating and human rating for a proportion of responses.

Problem

• Multimodal AI Systems
• Accessibility Planning
• Real World Places

Innovation

• multimodal AI
• accessibility planning
• claim verification
• visual evidence retrieval