Problem
Research questions and friction points this paper is trying to address.
OCR
Meta-Reasoning
Text-Rich Image Understanding
Multimodal Large Language Models
Reasoning Process Compliance
OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding
🤖 AI Summary
本文提出OCR-MetaReasoning基准，通过区分演绎、归纳和溯因三种推理方向来评估多模态大语言模型在文本丰富图像理解中的元推理能力。
📝 Abstract
Text-rich image understanding requires multimodal large language models (MLLMs) to organize OCR (Optical Character Recognition)-grounded evidence across words, layout, fields, charts, and visual correspondences. Existing evaluations often conflate extraction with reasoning and rarely test whether models follow the required reasoning direction: applying visible rules, abstracting hidden regularities, or recovering missing premises. We introduce OCR-MetaReasoning, a controlled single-image benchmark that treats deduction, induction, and abduction as distinct directions and separates final-answer correctness from reasoning-process compliance. The benchmark contains 1,500 verified samples in a balanced \(3\times5\) taxonomy crossing three reasoning types with five OCR-object categories, along with reference reasoning steps, automatic answer scoring, the Meta-Reasoning Macro Score (MRMS), and the Reasoning Process Compliance Score (RPCS). Experiments with representative closed-source and open-source MLLMs show that OCR-grounded meta-reasoning remains far from saturated: models struggle with visible-rule application and layout-sensitive inference, while process-compliant rationales can accompany incorrect final answers under exact-match evaluation. The code is available at https://github.com/gengxuli/OCR-MetaReasoning.
Innovation
Methods, ideas, or system contributions that make the work stand out.
OCR-MetaReasoning
multimodal large language models
Meta-Reasoning Macro Score (MRMS)
Reasoning Process Compliance Score (RPCS)
🔎 Similar Papers
No similar papers found.
