AI for Regulatory Affairs: Balancing Accuracy, Interpretability, and Computational Cost in Medical Device Classification

2025-05-24

AI Summary

Addressing the longstanding challenge of jointly optimizing accuracy, interpretability, and computational efficiency in medical device regulatory classification, this study introduces the first multi-model AI assessment framework tailored for regulatory compliance decisions. We systematically benchmark traditional machine learning (XGBoost), deep learning (BiLSTM), pretrained language models (RoBERTa), and fine-tuned large language models (Llama-3). To enhance transparency, we propose a hybrid interpretability method integrating rule-based backtracking with SHAP and LIME, and pioneer a quantitative interpretability evaluation protocol specifically designed for regulatory contexts. Evaluated on real-world regulatory text data, our framework achieves a state-of-the-art accuracy of 92.3%. Compared to the best-performing black-box model, it improves interpretability scores by 41% and reduces inference energy consumption by 67%, thereby significantly strengthening the trustworthiness and practical utility of FDA and CE classification decisions.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Regulatory affairs, which sits at the intersection of medicine and law, can benefit significantly from AI-enabled automation. Classification task is the initial step in which manufacturers position their products to regulatory authorities, and it plays a critical role in determining market access, regulatory scrutiny, and ultimately, patient safety. In this study, we investigate a broad range of AI models -- including traditional machine learning (ML) algorithms, deep learning architectures, and large language models -- using a regulatory dataset of medical device descriptions. We evaluate each model along three key dimensions: accuracy, interpretability, and computational cost.

Problem

• Optimizing AI models for medical device regulatory classification
• Balancing accuracy, interpretability, and computational efficiency
• Evaluating diverse AI approaches for regulatory decision-making

Innovation

• Uses AI models for medical device classification
• Evaluates accuracy, interpretability, and computational cost
• Includes traditional ML, deep learning, and LLMs