Leveraging artificial intelligence to detect ethical concerns in medical research: a case study

Area of Science:

• Medical Ethics
• Artificial Intelligence in Healthcare
• Clinical Research Administration

Background:

• Institutional Review Boards (IRBs) face challenges with research proposal approval delays due to staffing limitations.
• Artificial intelligence (AI), particularly large language models (LLMs), offers a potential solution to enhance IRB efficiency.
• LLMs can assist IRB members in streamlining the review process for research proposals.

Purpose of the Study:

• To evaluate the capability of four large language models (LLMs) in identifying ethical issues in clinical research scenarios.
• To assess the performance of LLMs in generating informed consent documents (ICDs).
• To determine the potential of LLMs as an adjunct tool for IRB review processes.

Main Methods:

• Four LLMs were tested using seven validated case studies.
• Queries focused on participant eligibility, vulnerability, informed consent, risk-benefit assessment, and placebo use.
• LLMs were also prompted to generate informed consent documents for the case scenarios.

Main Results:

• All four LLMs successfully responded to queries across all seven cases, with generally consistent outputs.
• LLMs showed suboptimal performance with single prompts for placebo suitability, risk mitigation, and potential participant risks.
• Multiple prompts improved LLM performance in identifying risks and optimizing placebo use.
• AI-generated ICDs included all fundamental elements but sometimes contained jargon, understated benefits, or omitted potential risks.

Conclusions:

• LLMs demonstrate potential in enhancing the identification of ethical issues in clinical research.
• LLMs can serve as a valuable adjunct tool for prescreening research proposals.
• The integration of LLMs may significantly improve the overall efficiency of Institutional Review Boards.