用大型语言模型评估队列研究的偏差风险
Danni Xia1,2, Honghao Lai1,2, Weilong Zhao1,2
1Department of Health Policy and Management, School of Public Health, https://ror.org/01mkqqe32Lanzhou University, Lanzhou, China.
Research synthesis methods
|February 2, 2026
概括
大型语言模型 (LLM) 显示出在队列研究中评估偏差风险 (ROB) 的前景,提供高效和准确的评估. 在这项试点研究中,ChatGPT-4o和Moonshot-v1-128k表现出明显的优势.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 生物统计学和流行病学
背景情况:
- 评估偏差风险 (ROB) 对队列研究的系统审查至关重要.
- 手动ROB评估是耗时和资源密集的.
- 大型语言模型 (LLM) 为自动化复杂的分析任务提供了潜力.
研究的目的:
- 评估在队列研究中使用LLMs进行ROB评估的可行性和准确性.
- 为了比较ChatGPT-4o,Moonshot-v1-128k和DeepSeek-V3在ROB评估中的性能.
- 评估基于LLM的ROB评估的效率和一致性.
主要方法:
- 一项试点研究涉及30项随机从Cochrane审查参考清单中选择的队列研究.
- 开发一个结构化的提示,以指导三个LLM (ChatGPT-4o,Moonshot-v1-128k,DeepSeek-V3) 在评估ROB时.
- 将LLM评估与三位基于证据的医学专家建立的黄金标准进行比较,使用正确评估率,灵敏度,特异性,F1分数和科恩的kappa等指标.
主要成果:
- 总体而言,LLMs的准确性是可以比较的 (80.8%-83.3%).
- 月球射击-v1-128k显示出对人口选择偏差的优越灵敏度和F1分数.
- 聊天GPT-4o表现出最高的一致性 (平均卡帕96.5%) 和明显更快 (97.3%的时间节省) 比手动评估.
结论:
- 包括ChatGPT-4o,Moonshot-v1-128k和DeepSeek-V3在内的LLM可以在队列研究中高效准确地评估ROB.
- 特定的LLM在ROB评估的不同方面表现出色 (例如,Moonshot用于人口选择,ChatGPT-4o用于一致性和速度).
- 法律法规显示了提高系统审查流程效率和准确性的巨大潜力.
相关概念视频
Bias in Epidemiological Studies
1.4K
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
1.4K
Confirmation Biases
8.2K
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
8.2K
Hindsight Biases
4.3K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
4.3K
Language
913
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
913
Bias
7.4K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
7.4K
Design Example: Analyzing Capacity Contours for Flood Risk Assessment
326
Flood risk assessment involves careful planning and analysis to ensure the safety of communities near water retention structures. Capacity contours are a vital tool in this process, as they illustrate the potential spread of water at specific levels in a given area. In the context of building a bund across a small valley, these contours play a critical role in evaluating the safety of nearby residential areas.In this example, the bund is intended to store stormwater in the valley. The engineers...
326


