MapTab: Can MLLMs Master Constrained Route Planning?

2026-02-20

AI Summary

Existing benchmarks struggle to effectively evaluate the reasoning capabilities of multimodal large language models (MLLMs) under complex constraints, particularly in path-planning tasks that require integrating visual maps with structured tabular data. To address this gap, this work proposes MapTab—the first multimodal evaluation benchmark that combines map images with attribute-rich tables containing temporal, pricing, and other real-world factors. MapTab encompasses subway networks across 160 global cities and 168 tourist attractions, incorporating four practical constraints: time, cost, comfort, and reliability. Based on a large-scale, human-annotated multimodal question set, our evaluation across 15 prominent MLLMs reveals that current models perform poorly under constrained perception, with multimodal fusion often underperforming unimodal baselines—highlighting significant deficiencies in cross-modal alignment and constraint-aware reasoning.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Systematic evaluation of Multimodal Large Language Models (MLLMs) is crucial for advancing Artificial General Intelligence (AGI). However, existing benchmarks remain insufficient for rigorously assessing their constrained reasoning capabilities. To bridge this gap, we introduce MapTab, a multimodal benchmark specifically designed to evaluate constrained reasoning in MLLMs via route planning tasks. MapTab requires MLLMs to perceive and ground visual cues from map images alongside route attributes (e.g., Time, Price) from structured tabular data. The benchmark encompasses two scenarios: Metromap, covering metro networks in 160 cities across 52 countries, and Travelmap, depicting 168 representative tourist attractions from 19 countries. In total, MapTab comprises 328 images, 196,800 route planning queries, and 3,936 QA queries, all incorporating 4 key constraints: Time, Price, Comfort, and Reliability. Extensive evaluations across 15 representative MLLMs reveal that current models face substantial challenges in constrained multimodal reasoning. Notably, under conditions of limited visual perception, multimodal collaboration often underperforms compared to unimodal approaches. We believe MapTab provides a challenging and realistic testbed to advance the systematic evaluation of MLLMs.

Problem

• constrained reasoning
• multimodal large language models
• route planning
• benchmark evaluation
• multimodal reasoning

Innovation

• constrained reasoning
• multimodal large language models
• route planning
• benchmark
• visual-tabular grounding