Standard Applicability Judgment and Cross-jurisdictional Reasoning: A RAG-based Framework for Medical Device Compliance

2025-06-23

AI Summary

In medical device regulatory compliance, determining the applicability of standards across jurisdictions (e.g., U.S. and China) relies heavily on expert judgment, yet faces challenges including fragmented documentation, high heterogeneity, and poor traceability. This paper proposes the first end-to-end standard applicability reasoning framework, integrating retrieval-augmented generation (RAG) with a domain-fine-tuned large language model (LLM), grounded in an expert-annotated international benchmark dataset. The framework enables region-aware cross-jurisdictional standard identification, conflict resolution, and traceable justification. Evaluated against retrieval-only, zero-shot, and rule-based baselines, it achieves 73% accuracy in standard classification and 87% top-5 retrieval recall—demonstrating substantial improvements in automated regulatory reasoning and jurisdictional adaptability.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Reasoning under Uncertainty: Applications
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Audits of web technologies, standards, platforms, and applications

Abstract

Identifying the appropriate regulatory standard applicability remains a critical yet understudied challenge in medical device compliance, frequently necessitating expert interpretation of fragmented and heterogeneous documentation across different jurisdictions. To address this challenge, we introduce a modular AI system that leverages a retrieval-augmented generation (RAG) pipeline to automate standard applicability determination. Given a free-text device description, our system retrieves candidate standards from a curated corpus and uses large language models to infer jurisdiction-specific applicability, classified as Mandatory, Recommended, or Not Applicable, with traceable justifications. We construct an international benchmark dataset of medical device descriptions with expert-annotated standard mappings, and evaluate our system against retrieval-only, zero-shot, and rule-based baselines. The proposed approach attains a classification accuracy of 73% and a Top-5 retrieval recall of 87%, demonstrating its effectiveness in identifying relevant regulatory standards. We introduce the first end-to-end system for standard applicability reasoning, enabling scalable and interpretable AI-supported regulatory science. Notably, our region-aware RAG agent performs cross-jurisdictional reasoning between Chinese and U.S. standards, supporting conflict resolution and applicability justification across regulatory frameworks.

Problem

• Automates medical device regulatory standard applicability determination
• Addresses fragmented documentation across different jurisdictions
• Enables cross-jurisdictional reasoning between Chinese and U.S. standards

Innovation

• RAG pipeline automates standard applicability determination
• Large language models infer jurisdiction-specific applicability
• Region-aware RAG enables cross-jurisdictional reasoning