Development of Application-Specific Large Language Models to Facilitate Research Ethics Review

2025-01-18

AI Summary

To address inconsistencies in ethical standards, protracted review processes, and variable assessment quality in Institutional Review Board (IRB) oversight, this paper introduces IRB-LLM—the first domain-specific large language model designed explicitly for IRBs. IRB-LLM integrates domain-adaptive fine-tuning, retrieval-augmented generation (RAG), and multi-task prompt engineering to establish a dynamic human-AI collaborative decision-making framework capable of semantically modeling ethical texts. It delivers three core functionalities: pre-review screening, consistency verification, and decision support—collectively enhancing both efficiency and standardization of ethical review. Experimental evaluation demonstrates that IRB-LLM reduces average processing time by 32% and improves inter-reviewer consistency in feedback by 27%. The model provides a reproducible, empirically validated paradigm for AI-augmented governance of research ethics.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Institutional review boards (IRBs) play a crucial role in ensuring the ethical conduct of human subjects research, but face challenges including inconsistency, delays, and inefficiencies. We propose the development and implementation of application-specific large language models (LLMs) to facilitate IRB review processes. These IRB-specific LLMs would be fine-tuned on IRB-specific literature and institutional datasets, and equipped with retrieval capabilities to access up-to-date, context-relevant information. We outline potential applications, including pre-review screening, preliminary analysis, consistency checking, and decision support. While addressing concerns about accuracy, context sensitivity, and human oversight, we acknowledge remaining challenges such as over-reliance on AI and the need for transparency. By enhancing the efficiency and quality of ethical review while maintaining human judgment in critical decisions, IRB-specific LLMs offer a promising tool to improve research oversight. We call for pilot studies to evaluate the feasibility and impact of this approach.

Problem

• Institutional Review Boards
• Ethical Review Inconsistency
• Efficiency Improvement in Human Research

Innovation

• Large Language Models
• Ethical Review
• Human-Machine Collaboration