为机构审查委员会进行预审开发人工智能工具:关于ChatGPT准确性和可重复性的试点研究
Yasuko Fukataki1,2, Wakako Hayashi1, Naoki Nishimoto3
1Health Data Science, Department of Social Science, Graduate School of Medicine, Hokkaido University, Sapporo, Japan.
PLOS digital health
|June 30, 2025
概括
生成型人工智能 (AI) 在协助对日本临床研究文件的伦理审查方面表现有前途. 定制提示提高了AI准确性和一致性,提取关键的伦理信息,支持机构审查委员会 (IRB) 流程.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 临床研究伦理学
背景情况:
- 临床研究文件的伦理评估对于参与者保护至关重要.
- 现有的AI工具在分析复杂的,特定语言的研究文档方面存在局限性.
- 生成型人工智能,如GPT-4和GPT-4o,为增强文档分析提供了潜力.
研究的目的:
- 评估生成AI模型 (GPT-4和GPT-4o) 在分析日语临床研究文件时的基线性能.
- 确定AI在从协议和知情同意表格 (ICF) 中提取伦理相关信息的准确性和一致性.
- 为了比较定制与标准提示对AI分析的有效性.
主要方法:
- 使用定制的OpenAI代理来评估GPT-4和GPT-4o的试点研究.
- 使用定制日语提示符的GPTs与使用GPT-4o的标准提示符的比较.
- 评估用于研究目标的信息提取,背景,设计,风险和益处.
- 在10个试验中测试了可重复性.
主要成果:
- GPT-4o在研究目标/背景方面达到了80%的协议,在研究设计方面达到了100%的协议.
- 这两种模型在多个试验中都显示出高可重现性.
- 与标准提示相比,定制的日本提示产生了更准确和更一致的AI输出.
结论:
- 生成型人工智能,特别是具有优化的提示,在支持机构前审查委员会 (IRB) 审查任务方面显示出潜在的实用性.
- 这项研究为未来的AI验证和临床研究伦理学标准化提供了基础数据.
- 人工智能工具旨在增强,而不是取代人类的伦理审查和IRB决策.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
592
05:33Introduction of an Integrated Pathology Image Management, Artificial Intelligence, and Reporting System
Published on: July 11, 2025
269
相关概念视频
02:07Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
01:16Non-equilibrium in the Cell
4.9K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.9K
01:56Ethics in Research
24.0K
Today, scientists agree that good research is ethical in nature and is guided by a basic respect for human dignity and safety. However, this has not always been the case. Modern researchers must demonstrate that the research they perform is ethically sound.
24.0K