体系的なレビューの方法論的質の評価を支援するために人工知能の使用
Manuel Marques-Cruz1, Filipe Pinto2, Rafael José Vieira3
1MEDCIDS - Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal; CINTESIS@RISE - Health Research Network, University of Porto, Porto, Portugal; Public Health Unit Marão e Douro Norte, Local Health Unit Trás-os-Montes e Alto Douro, Vila Real, Portugal.
Journal of clinical epidemiology
|August 27, 2025
まとめ
大規模な言語モデル (LLM) は,体系的なレビューの質を効率的に評価することができます. 精密調整されたGPT-3.5モデルは高い精度を示し,エビデンスに基づくガイドライン開発の改善の可能性を示しました.
科学分野:
- 医療における人工知能
- 証拠の統合
- 体系的なレビューの方法論
背景:
- 体系的なレビューの異質な方法論的質は意思決定に影響します.
- 大規模な言語モデル (LLM) は,効率的な品質評価の可能性を秘めています.
- 体系的なレビューの評価を簡素化することで,ガイドラインの作成に役立つ.
研究 の 目的:
- 体系的なレビューの方法論的質の評価のためのLLMベースのツールを開発し,評価する.
- このタスクにおけるLLMの正確性と信頼性を決定する.
主な方法:
- 100の体系的なレビューで8つのLLM (5つのベース,3つの微調整) を評価した.
- 品質評価のために27項目で検証されたReMarQツールを使用しました.
- 精度,カッパ,F1スコアを使用して,人間のレビュー者に対してLLMのパフォーマンスを比較した.
主要な成果:
- 微調整されたGPT-3.5モデルは高性能 (精度96.5%,カッパ0.90,F1-スコア0.91) を達成した.
- このモデルは,すべてのReMarQ項目の80%の精度と0.60kPaを超えました.
- 最優秀なLLMによる繰り返し評価では高い一貫性が見られた.
結論:
- LLMは,体系的なレビューの方法論的品質評価を正確にサポートする大きな可能性を示しています.
- LLMベースのツールは,エビデンス合成評価の効率と信頼性を高めることができます.
- この技術は,高品質な証拠を臨床ガイドラインに組み込むことを容易にする.
関連する概念動画
Quality Assurance
198
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
198
Systematic Error: Methodological and Sampling Errors
2.3K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
2.3K
Data Validation
234
Method validation is a crucial process in analytical chemistry designed to confirm that a given method consistently produces reliable and high-quality results. This process is essential when a method is applied to different sample matrices or when procedural modifications are made, ensuring that the results meet acceptable standards across various applications.
Key parameters for method validation include:
Key parameters for method validation include:
234
Qualitative Analysis
678
Qualitative analysis is the process of identifying elements, ions, or compounds in an unknown sample. It is the first and most fundamental type of analysis based on the hierarchy of analytical goals. This hierarchy is significant as it provides a structured approach to scientific research, with qualitative analysis serving as the initial step, providing essential information before moving on to quantitative or other forms of analysis.
There are two main approaches to qualitative analysis:...
There are two main approaches to qualitative analysis:...
678
The Scientific Method in Nursing Process
12.7K
The scientific method provides the foundation for any research. It is the most reliable and objective of all forms of gaining knowledge and guides in applying research-based evidence in practice and conducting future research.
When using research findings to change practice, one must understand the process used to guide a study. The scientific method is a systematic, step-by-step process that supports the data's validity, reliability, and generalizability. As a result, findings can be...
When using research findings to change practice, one must understand the process used to guide a study. The scientific method is a systematic, step-by-step process that supports the data's validity, reliability, and generalizability. As a result, findings can be...
12.7K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
174
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
174


