基于CONSORT报告准则的文本分类模型来评估随机对照试验出版物的完整性
Lan Jiang1, Mengfei Lan1, Joe D Menke1
1School of Information Sciences, University of Illinois Urbana-Champaign, 501 E Daniel Street, Champaign, IL, 61820, USA.
Scientific reports
|September 17, 2024
概括
开发文本分类模型有助于确保随机对照试验 (RCT) 出版物报告CONSORT检查清单项目. 一个微调的PubMedBERT模型在识别这些关键报告标准方面取得了最佳表现.
科学领域:
- 医疗信息学 医疗信息学
- 临床试验报告报告
- 自然语言处理自然语言处理.
背景情况:
- 随机对照试验 (RCT) 的完整和透明报告对于评估研究可信度至关重要.
- 康索特检查清单提供了报告RCT的基本指导方针.
- 需要自动化方法来有效评估遵守CONSORT报告标准.
研究的目的:
- 开发和比较文本分类模型,用于识别RCT出版物中的CONSORT检查列表项目.
- 评估数据增强技术对模型性能的影响.
- 评估特定部分模型的有效性,以改进CONSORT项目识别.
主要方法:
- 训练有素的句子分类模型使用PubMedBERT微调,BioGPT微调和GPT-4上下文学习在37个CONSORT项目注释的语料库上.
- 应用数据增强方法,包括Easy Data Augmentation (EDA),UMLS-EDA和基于GPT-4的文本重写.
- 开发和评估了特定部分的PubMedBERT模型 (例如,方法部分) 和一个包含句子上下文和部分标题的完整模型.
- 执行5倍交叉验证并报告性能指标,如精度,回忆,F1得分和AUC.
主要成果:
- 微调的PubMedBERT模型,利用句子与周围的上下文和部分标题,实现了最高的性能 (句子级别:0.71微F1;文章级别:0.90微F1).
- 数据增强方法对模型性能产生了有限的积极影响.
- 生物GPT微调和GPT-4上下文学习产生了低于最佳的结果.
- 一个特定于方法的模型改善了与方法相关的项目的识别,但其他特定于部分的模型并没有显著提高整体性能.
结论:
- 一个微调的PubMedBERT模型有效地识别了RCT出版物中的大多数CONSORT检查列表项目,尽管进一步的改进是可能的.
- 这些先进的模型可以支持期刊编辑工作流程和CONSORT合规性检查.
- 该研究强调了NLP在提高临床试验报告质量和透明度方面的潜力.
相关概念视频
Clinical Trials: Overview
2.9K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.9K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
124
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
124
Hazard Ratio
92
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
For example, in a clinical trial...
92
Clinical Trials
6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.6K
Classification of Illness
7.4K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.4K
Study Design in Statistics
7.9K
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
7.9K


