Reinforcement Learning for Tool-Calling Agents in Fast Healthcare Interoperability Resources (FHIR)

2026-05-13

AI Summary

This work addresses the challenge that existing agents struggle to perform accurate multi-hop clinical reasoning under the FHIR standard, often failing due to incorrect resource selection or violations of graph traversal constraints. To overcome this, the study introduces reinforcement learning into FHIR-based tool-calling agents for the first time, proposing an end-to-end post-training framework that formulates multi-step reasoning as a sequential decision-making problem over a structured knowledge graph. Integrating the CodeAct agent architecture with an LLM-based Judge reward mechanism grounded in execution outcomes, the approach achieves a significant improvement on FHIR-AgentBench: using the Qwen3-8B model, it raises answer accuracy from 50% (o4-mini) to 77%, substantially outperforming closed-source baselines while strictly adhering to healthcare data integrity constraints.

Abstract

Fast Healthcare Interoperability Resources (FHIR) is the dominant standard for interoperable exchange of healthcare data. In FHIR, electronic health records form a directed graph of resources. Answering clinically meaningful questions over FHIR requires agents to perform multi-step reasoning, filtering, and aggregation across multiple resource types. Prior work shows that even tool-augmented LLM agents (retrieval, code execution, multi-turn planning) often select the wrong resources or violate traversal constraints. We study this problem in the context of FHIR-AgentBench, a benchmark for realistic question answering over real-world hospital data, and frame reasoning on FHIR as a sequential decision-making problem over a queryable structured graph. We implement a multi-turn CodeAct agent and post-train it with reinforcement learning using a custom harness and tools. A LLM Judge provides execution-grounded rewards. Compared to prompt-based, closed-model baselines, RL post-training improves performance while enforcing data-integrity constraints. Empirically, our approach improves answer correctness from 50% (o4-mini) to 77% on FHIR-AgentBench using a smaller and cheaper Qwen3-8B model. We present an end-to-end post-training pipeline (environment building, harness construction, model training and custom evaluation) that reliably improves multi-turn reasoning over structured clinical graphs.

Problem

• FHIR
• tool-calling agents
• multi-step reasoning
• structured clinical graphs
• resource traversal constraints

Innovation

• Reinforcement Learning
• Tool-Calling Agents
• FHIR
• Structured Clinical Graphs
• CodeAct