在牙科中检索数值数据进行元分析的LLM的准确性
Vito Carlo Alberto Caponio1, Alejandro I Lorenzo-Pouso2, Marco Magalhaes3
1Department of Life Sciences, Health and Health Professions, Link Campus University, Via del Casale Di San Pio V 44, 00165, Rome, Italy; ORALMED Research Group, Department of Dental Clinical Specialties, School of Dentistry, Complutense University, 28040 Madrid, Spain.
Journal of dentistry
|November 20, 2025
概括
大型语言模型 (LLM) 在提取牙科系统性审查和元分析 (SRMA) 的单个数值结果方面显示出高准确性. 然而,在更高的数据聚合水平上,遗漏错误增加,限制了它们的独立使用.
科学领域:
- 牙科研究 牙科研究
- 证据综合研究
- 医疗保健中的人工智能
背景情况:
- 系统审查和元分析 (SRMA) 对基于证据的牙科至关重要.
- 在SRMA中提取数据是耗时且容易出现错误的.
- 大型语言模型 (LLM) 提供了自动化SRMA数据提取的潜力.
研究的目的:
- 评估四个LLM (DeepSeek v3 R1,Claude 3.5 Sonnet,ChatGPT-4o,Gemini 2.0-flash) 在提取牙科研究中的初级数值结果时的准确性.
- 为了比较不同LLM在SRMA数据提取中的性能.
主要方法:
- 通过API通过默认设置和SMART格式提示符查询LLM.
- 数据提取的准确性在子结果,结果和研究水平上进行了评估.
- 错误被归类为幻觉,错过的数据或遗漏.
主要成果:
- 在子结果层面,整体提取精度很高.
- 双子座2.0闪光灯的表现明显低于其他模型 (p < 0.01).
- 克劳德3.5索内特和DeepSeek-v3 R1在全文提取中表现出卓越的准确性和较低的省略率.
结论:
- 在牙科SRMA中,LLM显示出显著的数据提取潜力,但存在局限性.
- 准确性在模型之间有所不同,成本与性能无关.
- 标准化结果报告和准确,低成本的LLM可以提高证据合成效率.
相关概念视频
Mechanistic Models: Compartment Models in Individual and Population Analysis
230
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
230
Regression Toward the Mean
6.8K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.8K
Statistical Analysis: Overview
14.1K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
14.1K
Teeth
1.6K
The formation of teeth, also known as odontogenesis, is a complex process that begins in utero, around the sixth week of embryonic development. There are three stages to this process: the bud stage, the cap stage, and the bell stage.
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin...
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin...
1.6K


