Physics as the label for measuring and correcting materials reasoning in multimodal models

2026-09-10

AI Summary

论文提出MatPCR基准，通过物理定律验证多模态模型的材料推理一致性，无需人工标注。

Abstract

Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate physical law. Evaluation matches final answers to scarce human labels, while discovery agents verify final proposals or density functional theory (DFT) execution. Neither measures the physical consistency of a model's reasoning chain. Materials data carries its own physics, making a large class of materials reasoning verifiable without annotation. We introduce MatPCR, a label-free benchmark whose programmatic oracles check diffraction geometry through Bragg's law, scale bars, spectral peaks, and Materials Project-grounded checks of near-hull stability, computed band-gap class, and net magnetization. We define the Physical-Consistency Rate over image and structure inputs; introduce Constraint-Grounded Self-Verification, an agentic loop whose gain survives self-refinement and equal-compute re-prompting controls; release an open verifier useful in distribution but near chance on all six held-out constraint types; and derive an exact identity for how oracle error displaces the reported rate.

Problem

• materials reasoning
• physical consistency
• multimodal models

Innovation

• MatPCR
• Physical-Consistency Rate
• Constraint-Grounded Self-Verification
• Bragg's law