GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models

2025-11-14

AI Summary

Existing benchmarks typically evaluate discriminative understanding or free-form image generation in isolation, failing to characterize the integrated “comprehension–planning–construction” cognitive process underlying generative reasoning in unified multimodal models. To address this gap, we propose GGBench—the first benchmark designed specifically for evaluating geometric generative reasoning in unified multimodal models. It adopts natural-language-instructed, controllable, and precise 2D graphic construction as a unifying paradigm to systematically assess joint cross-modal understanding and generation capabilities. GGBench features hierarchically structured tasks that integrate semantic parsing, spatial reasoning, and controllable generation, enabling fine-grained diagnostic analysis of the generative process. Through standardized evaluation protocols and reproducible metrics, it fills a critical void in assessing active geometric construction ability, significantly improving the fidelity of evaluating generative cognition. GGBench thus establishes a more rigorous evaluation standard for next-generation embodied intelligent systems.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The advent of Unified Multimodal Models (UMMs) signals a paradigm shift in artificial intelligence, moving from passive perception to active, cross-modal generation. Despite their unprecedented ability to synthesize information, a critical gap persists in evaluation: existing benchmarks primarily assess discriminative understanding or unconstrained image generation separately, failing to measure the integrated cognitive process of generative reasoning. To bridge this gap, we propose that geometric construction provides an ideal testbed as it inherently demands a fusion of language comprehension and precise visual generation. We introduce GGBench, a benchmark designed specifically to evaluate geometric generative reasoning. It provides a comprehensive framework for systematically diagnosing a model's ability to not only understand and reason but to actively construct a solution, thereby setting a more rigorous standard for the next generation of intelligent systems. Project website: https://opendatalab-raiser.github.io/GGBench/.

Problem

• Evaluating unified multimodal models' generative reasoning capabilities
• Bridging the gap between discriminative understanding and image generation
• Assessing geometric construction requiring language and visual synthesis

Innovation

• Proposes geometric construction as generative reasoning testbed
• Introduces benchmark for multimodal geometric reasoning evaluation
• Systematically diagnoses language comprehension and visual generation