Are We Measuring Strategy or Phrasing? The Gap Between Surface- and Approach-Level Diversity in LLM Math Reasoning

2026-06-29

AI Summary

Existing evaluations of diversity in large language models for mathematical reasoning are largely confined to superficial lexical variations, failing to capture genuine diversity at the problem-solving strategy level. This work introduces, for the first time, the concept of “strategy-level diversity” and develops a human-calibrated evaluation framework to quantify this dimension. By incorporating a diversity-aware RLVR training approach and a strategy-diversity reward mechanism, the study reveals a systematic discrepancy between conventional diversity metrics and strategy-level diversity. The findings demonstrate that generating candidate solutions with diverse strategies substantially enhances test-time scaling performance; however, directly optimizing for strategy diversity risks overfitting to evaluator preferences.

Abstract

Diversity in LLM mathematical reasoning is critical for exploration, but common diversity metrics mostly capture surface-level variation rather than differences in how a problem is solved. We address this gap by introducing approach-level diversity: variation in strategies across correct solutions to the same problem. Using a human-calibrated LLM judge framework, we show that prior diversity measures are unreliable proxies for approach-level diversity, and this mismatch carries over to diversity-aware RLVR, where target metrics are preserved while approach-level diversity declines. Investigating when approach-level diversity helps and whether it can be directly induced, we find that approach-diverse candidate sets improve test-time scaling. However, optimizing an LLM judge diversity reward during training causes the policy to exploit judge-specific preferences rather than broaden its approaches, leaving direct optimization of approach-level diversity as an open problem. Together, our work introduces the notion of approach-level diversity and uncovers a systematic divergence between surface- and approach-level signals, marking a step toward LLMs that reason in genuinely diverse, human-like ways.

Problem

• approach-level diversity
• surface-level diversity
• LLM math reasoning
• diversity metrics
• reasoning strategies

Innovation

• approach-level diversity
• mathematical reasoning
• LLM diversity evaluation
• reinforcement learning with diversity
• reasoning strategies