CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving

2026-01-05

arXiv.org

AI Summary

This work addresses the challenge that existing multimodal large language models struggle to effectively integrate perceptual understanding with symbolic reasoning in visual mathematical tasks, often failing to faithfully leverage visual cues. To bridge this gap, the authors propose CogFlow—a human cognition-inspired three-stage framework (Perception → Internalization → Reasoning)—which explicitly introduces a knowledge internalization stage to establish a hierarchical flow from perception to reasoning. CogFlow is optimized through a synergistic combination of visual reward mechanisms, a knowledge internalization reward model, and a visual gating strategy, trained on MathCog, a newly curated high-quality dataset comprising over 120,000 samples. Experimental results demonstrate that CogFlow significantly outperforms current state-of-the-art methods across multiple visual mathematical reasoning benchmarks, substantially enhancing the model’s visually grounded reasoning capabilities.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Despite significant progress, multimodal large language models continue to struggle with visual mathematical problem solving. Some recent works recognize that visual perception is a bottleneck in visual mathematical reasoning, but their solutions are limited to improving the extraction and interpretation of visual inputs. Notably, they all ignore the key issue of whether the extracted visual cues are faithfully integrated and properly utilized in subsequent reasoning. Motivated by this, we present CogFlow, a novel cognitive-inspired three-stage framework that incorporates a knowledge internalization stage, explicitly simulating the hierarchical flow of human reasoning: perception$\Rightarrow$internalization$\Rightarrow$reasoning. Inline with this hierarchical flow, we holistically enhance all its stages. We devise Synergistic Visual Rewards to boost perception capabilities in parametric and semantic spaces, jointly improving visual information extraction from symbols and diagrams. To guarantee faithful integration of extracted visual cues into subsequent reasoning, we introduce a Knowledge Internalization Reward model in the internalization stage, bridging perception and reasoning. Moreover, we design a Visual-Gated Policy Optimization algorithm to further enforce the reasoning is grounded with the visual knowledge, preventing models seeking shortcuts that appear coherent but are visually ungrounded reasoning chains. Moreover, we contribute a new dataset MathCog for model training, which contains samples with over 120K high-quality perception-reasoning aligned annotations. Comprehensive experiments and analysis on commonly used visual mathematical reasoning benchmarks validate the superiority of the proposed CogFlow.

Problem

• visual mathematical reasoning
• perception-reasoning gap
• visual cue integration
• multimodal reasoning
• visually grounded reasoning

Innovation

• Knowledge Internalization
• Visual Mathematical Reasoning
• Synergistic Visual Rewards
• Visual-Gated Policy Optimization
• CogFlow