MedCalc-R1: Knowledge-Guided Reward Framework for Medical Mathematical Reasoning

2026-08-09

AI Summary

This work addresses the limitations of conventional tolerance-based reward mechanisms in medical mathematical reasoning, which suffer from difficulties in threshold calibration, training instability, and insufficient accuracy, thereby failing to meet stringent clinical safety requirements. To overcome these challenges, the authors propose a knowledge-guided hybrid reward framework (RLVR) that enhances interpretability by explicitly generating and validating computational formulas. The framework synergistically integrates hard clinical safety constraints with a precision-sensitive soft reward function to jointly optimize the model’s reasoning process. Experimental results demonstrate that the proposed method significantly outperforms existing baselines on medical mathematical reasoning tasks, achieving notable improvements in both reasoning accuracy and generalization while effectively balancing safety and performance.

Abstract

In Reinforcement Learning with Verifiable Rewards (RLVR) frameworks for mathematical reasoning tasks, floating-point results are typically evaluated using a tolerance-based reward. However, this strategy suffers from challenges such as difficulty in threshold calibration, unstable training dynamics, and limited accuracy, especially in clinical scenarios. To address these limitations, we propose a knowledge-guided hybrid reward framework (\textsc{MedCalc-R1}). Specifically, we introduce a knowledge verification reward mechanism that enforces explicit generation of computational formulas, which are further validated by an external verifier to enhance interpretability and reasoning reliability. Furthermore, we design a hybrid soft-hard reward scheme combining a hard constraint based on clinical safety thresholds with a soft, precision-sensitive reward that progressively guides learning within the acceptable range. Experimental results demonstrate that our method significantly outperforms existing baselines in both reasoning accuracy and generalization capability, validating the effectiveness and applicability in safety-critical domains.

Problem

• reinforcement learning
• mathematical reasoning
• floating-point evaluation
• clinical safety
• reward calibration

Innovation

• knowledge-guided reward
• medical mathematical reasoning
• hybrid reward scheme
• verifiable reasoning
• clinical safety constraints