CONSORT-TM:用于评估随机对照试验出版物完整性的文本分类模型
Lan Jiang1, Mengfei Lan1, Joe D Menke1
1School of Information Sciences, University of Illinois Urbana-Champaign, Champaign, IL, USA.
medRxiv : the preprint server for health sciences
|April 18, 2024
概括
在随机对照试验中开发人工智能模型来检查CONSORT报告准则是可行的. 一个微调的PubMedBERT模型显示了改善报告质量和手稿完整性的承诺.
科学领域:
- 医疗信息学医学信息学
- 临床试验报告报告
背景情况:
- 联合报告试验标准 (CONSORT) 准则旨在提高随机对照试验 (RCT) 出版物的质量.
- 检查CONSORT遵守的自动化方法可以简化编辑工作流程并提高报告准确性.
研究的目的:
- 开发和评估文本分类模型,以识别RCT出版物中CONSORT检查列表项的存在.
- 为了比较不同模型的性能,包括微调的PubMedBERT,BioGPT和GPT-4上下文学习.
- 评估数据增强技术对模型性能的影响.
主要方法:
- 一组RCT出版物在句子层面被注释为37个细粒度的CONSORT项目.
- 使用PubMedBERT微调,BioGPT微调和GPT-4上下文学习来训练句子分类模型.
- 应用了数据增强方法 (EDA,UMLS-EDA,GPT-4重写) 来处理小型训练数据集.
- 针对特定部分的PubMedBERT模型进行了微调,以评估特定手稿部分项目的性能.
主要成果:
- 微调的PubMedBERT模型,包括句子上下文和部分标题,实现了最佳性能 (0.71微F1,0.64宏F1).
- 数据增强提供了有限的好处,UMLS-EDA显示的结果略好于基于GPT-4的增强.
- 生物GPT微调和GPT-4上下文学习产生了低于最佳的性能.
- 方法特定模型改善了方法项的性能,但其他部分的部分特定模型没有显著影响整体结果.
结论:
- 微调的PubMedBERT模型可以有效地识别RCT出版物中的大多数CONSORT检查列表项目,尽管进一步的改进是可能的.
- 这些自动化模型可以支持期刊编辑流程和CONSORT合规性检查.
- 这些人工智能驱动的工具可以促进改进报告质量和手稿的完整性.
更多相关视频
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
15.9K
04:09Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
Published on: October 10, 2018
8.2K
相关概念视频
Clinical Trials: Overview
2.9K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.9K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
126
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
126
Clinical Trials
6.7K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.7K
Randomized Experiments
6.9K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.9K
Classification of Illness
7.5K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.5K
Study Design in Statistics
8.0K
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
8.0K
