评估ChatGPT模型版本的准确性,以提供寻求护理的建议
Marvin Kopka1, Longqi He2, Markus A Feufel2
1Division of Ergonomics, Department of Psychology and Ergonomics (IPA), Technische Universität Berlin, Berlin, Germany. marvin.kopka@tu-berlin.de.
Communications medicine
|February 24, 2026
概括
聊天GPT模型在寻求护理建议方面显示出有限的准确性. 虽然新版本提供了自我护理建议,但它们的可靠性不足以独立使用. 聚合输出可以提高绩效.
科学领域:
- 医疗保健中的人工智能
- 医疗决策支持系统 医疗决策支持系统
- 自然语言处理自然语言处理.
背景情况:
- 非专业人士越来越多地使用像ChatGPT这样的AI工具来做健康决策.
- 人工智能产生的寻求护理建议的准确性,特别是来自较新的模型,需要进行彻底的评估.
- 现有的研究还没有全面评估所有可用的ChatGPT模型的医疗建议准确性.
研究的目的:
- 评估目前所有可用的ChatGPT模型提供的寻求护理建议的准确性.
- 为了比较不同版本的ChatGPT在医疗紧急情况分类中的性能.
- 探索改善人工智能产生的医疗建议准确性的方法.
主要方法:
- 评估了22个ChatGPT模型,使用了45个验证的标签,每个标签被提示了10次 (共9900次评估).
- 紧急情况分为紧急护理,非紧急护理或自我护理类别.
- 根据两名医生建立的黄金标准验证了AI建议,并测试了聚合算法.
主要成果:
- o1-迷你模型以74%的精度显示出最高的精度.
- 在较新的ChatGPT模型中没有观察到一致的准确性改善.
- 推理模型显示自我护理案例识别得到了改进;汇总输出的准确性提高了4%.
- 在多个试验中选择最低的紧急级别提高了整体准确性.
结论:
- 目前的ChatGPT模型在寻求护理的决策中无法独立使用,因为不够准确.
- 新的AI模型越来越多地提供自我护理建议,但可靠性仍然是一个问题.
- 使用聚合算法来利用输出可变性可以显著提高现有的AI模型的性能.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.7K
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
8.2K
相关概念视频
01:28Accuracy, limits, and approximation
1.3K
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
1.3K
02:07Improving Translational Accuracy
15.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
15.3K
02:07Improving Translational Accuracy
3.7K
3.7K
01:13Accuracy and Errors in Hypothesis Testing
634
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
634
01:03Prediction Intervals
3.5K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
3.5K
01:52Accuracy and Precision
16.3K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate...
16.3K