标准2Query 3.0:利用生成型大语言模型进行临床试验资格查询生成
Jimyung Park1, Yilu Fang1, Casey Ta1
1Department of Biomedical Informatics, Columbia University, New York, United States.
Journal of biomedical informatics
|May 2, 2024
概括
Criteria2Query (C2Q) 3.0使用GPT-4将临床试验资格标准转换为数据库查询,提高患者识别的准确性. 需要进一步的研究,以确保临床研究中的大型语言模型可靠性.
科学领域:
- 临床信息学 临床信息学
- 人工智能在医学中的应用
- 健康 数据科学 数据科学
背景情况:
- 对于临床研究而言,自动化患者识别是一个重大挑战.
- 现有的将资格标准转化为数据库查询的方法通常是手动的,耗时的.
- 标准2Query (C2Q) 系统旨在简化这一过程.
研究的目的:
- 引入和评估Criteria2Query (C2Q) 3.0,一个使用GPT-4的系统,用于将临床试验资格标准半自动转换为可执行的数据库查询.
- 评估GPT-4在概念提取和SQL查询生成中对临床试验资格标准的性能.
主要方法:
- C2Q 3.0使用了三个GPT-4提示符来提取概念,生成SQL查询和推理.
- 概念提取与20项临床试验的手册注释进行了比较.
- 域专家对临床试验子集的SQL生成准确性和推理质量进行了评估.
主要成果:
- 在518个概念中,GPT-4获得了F1得分0.891的概念提取.
- 该系统在SQL生成中发现了29个错误,逻辑错误是最常见的 (34.48%).
- 推理评估显示了高的一致性 (平均4.70) 和有用性 (平均4.37),但读取性较低 (平均3.95).
结论:
- 在C2Q 3.0系统中,GPT-4显著提高了提取临床试验资格标准概念的准确性.
- 这些发现强调了大型语言模型在临床研究数据管理中的潜力.
- 需要继续进行调查,以充分确定临床应用的大型语言模型的可靠性.
相关概念视频
Genetic Lingo
Overview
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
ER Retrieval Pathway
In the secretory pathway, vesicles transport proteins from one cellular compartment to another in forward transport to deliver the protein to its correct location. Occasionally, misfolded proteins and incorrect proteins escape their original compartments, and a retrieval pathway is used to return the escaped proteins to their original compartment.
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Clinical Trials
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
Clinical Trials: Overview
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...


