Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training

2025-05-01

AI Summary

To address the scarcity of risk-classification annotations and the limitations of unimodal approaches in medical device regulation, this paper proposes a cross-modal self-training framework integrating textual and visual modalities. Methodologically, we design a multimodal Transformer with cross-modal cross-attention to model semantic alignments between text and images, coupled with iterative pseudo-labeling self-training to alleviate the small-sample bottleneck; model outputs are further calibrated via support vector machine (SVM). To our knowledge, this is the first work synergizing self-training with multimodal Transformers for automated regulatory risk classification. Evaluated on a real-world regulatory dataset, our method achieves 90.4% accuracy and 97.9% AUROC—outperforming unimodal baselines by 13.2 and 35.6 percentage points, respectively—with self-training contributing a 3.3-point gain. The approach significantly enhances generalization under limited supervision and improves clinical deployability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Accurate classification of medical device risk levels is essential for regulatory oversight and clinical safety. We present a Transformer-based multimodal framework that integrates textual descriptions and visual information to predict device regulatory classification. The model incorporates a cross-attention mechanism to capture intermodal dependencies and employs a self-training strategy for improved generalization under limited supervision. Experiments on a real-world regulatory dataset demonstrate that our approach achieves up to 90.4% accuracy and 97.9% AUROC, significantly outperforming text-only (77.2%) and image-only (54.8%) baselines. Compared to standard multimodal fusion, the self-training mechanism improved SVM performance by 3.3 percentage points in accuracy (from 87.1% to 90.4%) and 1.4 points in macro-F1, suggesting that pseudo-labeling can effectively enhance generalization under limited supervision. Ablation studies further confirm the complementary benefits of both cross-modal attention and self-training.

Problem

• Automating medical device risk classification for regulatory oversight
• Integrating text and visual data for accurate device classification
• Improving model generalization with self-training under limited supervision

Innovation

• Transformer-based multimodal framework for classification
• Cross-attention mechanism captures intermodal dependencies
• Self-training strategy enhances generalization under limited supervision