Mat-Pref: Verifiable-Reward Training Improves Compositional Reasoning in Inorganic Materials

2026-06-19

AI Summary

Existing scientific reasoning benchmarks struggle to disentangle a model’s capabilities in structural generalization, property transfer, and memorization for inorganic material composition reasoning, and lack fine-grained evaluation mechanisms. This work introduces Mat-Pref, a new benchmark comprising 10,837 ion-substitution questions derived from the Materials Project, enabling separable assessment through three task types: in-distribution prediction, out-of-structure-family extrapolation, and cross-property transfer. Combining supervised fine-tuning with Group Relative Policy Optimization (GRPO), Qwen3-8B achieves 65.2% accuracy in-distribution and 71.6% on unseen structure families—surpassing zero-shot Qwen3-235B by over 20 percentage points—and narrows the gap between lenient and strict scoring by 9.7 points. GRPO substantially enhances the reliability and consistency of compositional reasoning, enabling, for the first time, decoupled evaluation of structural generalization and property transfer.

Abstract

Reinforcement learning from verifiable rewards (RLVR) has driven rapid progress in mathematical and code reasoning, but when extended to science, existing benchmarks do not decompose what generalizes: do gains reflect structural transfer, property transfer, or memorization? We introduce Mat-Pref, a benchmark of 10,837 ionic-substitution questions across 11 inorganic structure families, grounded in density functional theory calculations from the Materials Project, with three evaluation splits that isolate in-distribution performance, generalization to entirely held-out structure families, and cross-property transfer: applying band-gap reasoning to hosts seen during training only through formation-energy supervision. Four zero-shot frontier models (70-671B parameters) remain in the 33-54% range on every split, confirming that scale alone does not resolve the compositional chemical reasoning this task demands. A two-stage pipeline of supervised fine-tuning followed by Group Relative Policy Optimization (GRPO) lifts Qwen3-8B to 65.2% in-distribution and 71.6% on held-out families, exceeding zero-shot Qwen3-235B by over 20 percentage points on both structural-generalization splits. Self-consistency sampling shows that the SFT policy can already produce correct answers but cannot reliably surface them as the modal response; GRPO reshapes the distribution so that correct answers become modal rather than merely reachable, and this sharper commitment is visible mechanistically: logit lens analysis reveals a ${\sim}$20pp advantage in answer crystallization at the critical decision layer. We formalize this observation as a distractor-permutation consistency metric under which GRPO narrows the gap between lenient scoring (at least one permutation correct) and strict scoring (all permutations correct) from 24.0 to 14.3 percentage points.

Problem

• compositional reasoning
• inorganic materials
• generalization
• verifiable rewards
• benchmark

Innovation

• verifiable-reward training
• compositional reasoning
• Group Relative Policy Optimization
• structural generalization
• distractor-permutation consistency