在RCT中使用GPT? 使用人工智能来确定遵守临床试验报告准则
James G Wrightson1, Paul Blazey2, David Moher3
1Department of Physical Therapy, The University of British Columbia Faculty of Medicine, Vancouver, British Columbia, Canada.
BMJ open
|March 19, 2025
概括
大型语言模型 (AI-LLMs) 在评估临床试验报告准则的遵守方面表现有前途. 在体育医学试验中,GPT-4实现了90%的准确性,而微调的Llama 2在体育医学试验中达到83%的准确性.
科学领域:
- 医学研究方法论医学研究方法论.
- 医疗保健中的人工智能
- 临床试验报告标准
背景情况:
- 遵守报告准则对于临床试验质量至关重要.
- 以前试图改善坚持的尝试产生了不一致的结果.
- AI-LLMs提供了一个潜在的新方法来评估报告准则的合规性.
研究的目的:
- 评估AI-LLMs在确定报告准则合规性的准确性.
- 评估OpenAI GPT-4和Meta Llama 2模型的性能.
- 分析体育医学和运动科学临床试验报告的合规性.
主要方法:
- 对113份临床试验报告的回顾性数据分析.
- 使用了OpenAI GPT-4 Turbo,GPT-4 Vision和Meta Llama 2 70B的模型. 这是一个非常好的选择.
- 模型被要求回答报告指南问题;Llama 2进行了微调.
主要成果:
- 对于文章文本,GPT-4 Turbo实现了90%的准确性 (F1分数=0.89).
- 微调的Llama 2实现了83%的准确性 (F1得分=0.84).
- 对于参与者流程图,GPT-4视觉实现了100%的准确性,但在缺失的细节方面遇到了困难 (57%的准确性).
结论:
- 无论是GPT-4还是精细调整的Llama 2,都显示出评估报告准则遵守的潜力.
- 建议进一步开发高效的,开源的AI-LLM.
- 提高AI-LLM在这个领域的准确性的方法需要进一步探索.
更多相关视频
08:36Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
413
06:05The Participant-Reported Implementation Update and Score PRIUS: A Novel Method for Capturing Implementation-Related Data Over Time
Published on: February 19, 2021
1.2K
相关概念视频
01:16Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
01:20Introduction to Documentation and Reporting
1.9K
Documentation is the systematic process of formally recording, maintaining, and communicating information.
Nursing documentation records essential information and details regarding a patient's care and treatment in written or electronic form. It is a critical aspect of nursing practice that involves documenting assessments, interventions, outcomes, and other relevant details about a patient's health status.
Documentation maps the patient's health journey by creating a comprehensive...
Nursing documentation records essential information and details regarding a patient's care and treatment in written or electronic form. It is a critical aspect of nursing practice that involves documenting assessments, interventions, outcomes, and other relevant details about a patient's health status.
Documentation maps the patient's health journey by creating a comprehensive...
1.9K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
110
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
110
01:30Guidelines for Nursing Documentation I
1.0K
Quality documentation and reporting share essential characteristics that ensure they are practical and valuable resources for those who use them. These characteristics are:
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
1.0K
01:24Data Reporting and Recording
4.6K
Reporting and recording are crucial in data documentation. The timely, thorough, and accurate documentation of facts is essential when recording patient data. Failure to record findings during an assessment or interpretation of a problem will result in loss of information and make the patient document unreliable. The reader is left with general impressions if the information is not specific. A recording is documenting data of the individual's health information in a traceable, secure, and...
4.6K
02:07Blind Procedures
10.6K
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which...
10.6K