系統的レビューの方法論的質の評価は,生成的な大規模な言語モデルを用いて行われます
Bowen Yao1,2, Onuralp Ergun1,2, Maylynn Ding2
1Minneapolis VA Healthcare System, Minneapolis, MN, United States.
まとめ
系統的レビュー (SR) の質を評価する可能性を示しています. 特定の指示により,GPTは品質評価の93%の精度を達成し,効率的で信頼性の高い評価能力を示しました.
科学分野:
- 医療における人工知能
- 医療情報学
- 泌尿器科の研究
背景:
- 体系的なレビュー (SR) の方法論的質の評価は,エビデンスベースの医学にとって極めて重要です.
- 複合的な分析タスクを自動化する可能性を秘めている.
研究 の 目的:
- 泌尿器科のSRの方法論的質の評価における生成的LLMの精度を評価する.
- LLMベースの品質評価と人間の専門家評価を比較する.
主な方法:
- 114の泌尿器学的SRは,ヒトの専門家とカスタマイズされたGPTモデルによって評価されました.
- GPTは3回のゼロショット評価とチェーン・オブ・思考の誘導を用いた強化された試験を受けました.
- 性能指標には,精度,感度,特異性,および人間の判断に対するF1スコアが含まれています.
主要な成果:
- GPTは,ヒトの審査者との全体的な一致度が75%に達し,重要な基準は77%であった.
- 平均F1スコアは0. 66で,内部有効性は85%でした.
- クリティカル基準の一致性を91%に,全体的な精度を93%に改善しました.
結論:
- 生成性LLMは,泌尿器科におけるSRの効率的で正確な品質評価のための有望な能力を示しています.
- LLMベースのツールは,見直しプロセスを効率化し,証拠の合成をサポートすることができます.
関連する概念動画
Qualitative Analysis
678
Qualitative analysis is the process of identifying elements, ions, or compounds in an unknown sample. It is the first and most fundamental type of analysis based on the hierarchy of analytical goals. This hierarchy is significant as it provides a structured approach to scientific research, with qualitative analysis serving as the initial step, providing essential information before moving on to quantitative or other forms of analysis.
There are two main approaches to qualitative analysis:...
There are two main approaches to qualitative analysis:...
678
Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
223
Drug disposition in the body is a complex process and can be studied using two major approaches: the model and the model-independent approaches.
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
223
Improving Translational Accuracy
2.7K
2.7K
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
126
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
126
Systematic Error: Methodological and Sampling Errors
2.3K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
2.3K
Language Development
444
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
444


