使用生成性大型语言模型评估系统性审查的方法质量
Bowen Yao1,2, Onuralp Ergun1,2, Maylynn Ding2
1Minneapolis VA Healthcare System, Minneapolis, MN, United States.
概括
产生性大型语言模型 (LLM) 显示了评估系统审查 (SR) 质量的潜力. 通过特定的指令,GPT在质量评估中获得了93%的准确性,这表明了高效可靠的评估能力.
科学领域:
- 医学的人工智能
- 医疗信息学
- 泌尿学研究
背景情况:
- 评估系统性审查的方法质量对于基于证据的医学至关重要.
- 生成型大语言模型 (LLM) 提供了复杂分析任务的自动化潜力.
研究的目的:
- 评估生成性LLM在评估泌尿学SR的方法质量的准确性.
- 将基于LLM的质量评估与人类专家评估进行比较.
主要方法:
- 人类专家和定制的GPT模型评估了114个泌尿器官SR.
- GPT经历了三次零射击评估代和使用思维链提示的增强试验.
- 性能指标包括精度,灵敏度,特异性和F1比人类判断的得分.
主要成果:
- 总体而言,GPT与人类审查者达到了75%的一致性,关键标准达到了77%.
- 平均F1评分为0.66%,内部有效率高达85%.
- 提升了对关键标准的一致性,达到91%,整体准确性达到93%.
结论:
- 生成性LLM显示出在泌尿学中有效和准确的质量评估的能力.
- 基于LLM的工具可能会简化审查过程并支持证据综合.
相关概念视频
Qualitative Analysis
678
Qualitative analysis is the process of identifying elements, ions, or compounds in an unknown sample. It is the first and most fundamental type of analysis based on the hierarchy of analytical goals. This hierarchy is significant as it provides a structured approach to scientific research, with qualitative analysis serving as the initial step, providing essential information before moving on to quantitative or other forms of analysis.
There are two main approaches to qualitative analysis:...
There are two main approaches to qualitative analysis:...
678
Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
223
Drug disposition in the body is a complex process and can be studied using two major approaches: the model and the model-independent approaches.
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
223
Improving Translational Accuracy
2.7K
2.7K
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
126
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
126
Systematic Error: Methodological and Sampling Errors
2.3K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
2.3K
Language Development
444
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
444


