GeometryZero: Improving Geometry Solving for LLM with Group Contrastive Policy Optimization

2025-06-08

AI Summary

In geometric problem solving, two major bottlenecks persist: the difficulty of judging auxiliary constructions and the limited reasoning capacity of lightweight models. To address these, we propose a novel reinforcement learning paradigm for lightweight LLMs grounded in verifiable rewards. Our method introduces a group-wise contrastive masking mechanism that dynamically assigns positive/negative construction rewards based on contextual cues; incorporates a chain-length reward to encourage complete, stepwise reasoning; and jointly optimizes auxiliary construction generation, geometric theorem verification, and policy refinement. The resulting Group Contrastive Policy Optimization (GCPO) framework achieves an average 4.29% improvement over state-of-the-art baselines—including GRPO—on benchmarks such as Geometry3K and MathVista. Notably, GCPO enables low-cost models to approach the geometric reasoning performance of GPT-4o, marking the first approach to realize autonomous, verifiable, interpretable, and computationally efficient auxiliary construction decisions.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recent advances in large language models (LLMs) have demonstrated remarkable capabilities across diverse domains, particularly in mathematical reasoning, amid which geometry problem solving remains a challenging area where auxiliary construction plays a enssential role. Existing approaches either achieve suboptimal performance or rely on massive LLMs (e.g., GPT-4o), incurring massive computational costs. We posit that reinforcement learning with verifiable reward (e.g., GRPO) offers a promising direction for training smaller models that effectively combine auxiliary construction with robust geometric reasoning. However, directly applying GRPO to geometric reasoning presents fundamental limitations due to its dependence on unconditional rewards, which leads to indiscriminate and counterproductive auxiliary constructions. To address these challenges, we propose Group Contrastive Policy Optimization (GCPO), a novel reinforcement learning framework featuring two key innovations: (1) Group Contrastive Masking, which adaptively provides positive or negative reward signals for auxiliary construction based on contextual utility, and a (2) length reward that promotes longer reasoning chains. Building on GCPO, we develop GeometryZero, a family of affordable-size geometric reasoning models that judiciously determine when to employ auxiliary construction. Our extensive empirical evaluation across popular geometric benchmarks (Geometry3K, MathVista) demonstrates that GeometryZero models consistently outperform baselines (e.g. GRPO), achieving an average improvement of 4.29% across all benchmarks.

Problem

• Improving geometry problem solving in LLMs with auxiliary construction
• Reducing computational costs by training smaller models effectively
• Addressing limitations of unconditional rewards in geometric reasoning

Innovation

• Group Contrastive Policy Optimization for rewards
• Adaptive reward signals for constructions
• Length reward for reasoning chains