由基于AI的症状检查器开发的差异诊断列表的诊断准确度的纵向变化:回顾性观察性研究
Yukinori Harada1,2, Tetsu Sakamoto1, Shu Sugimoto3
1Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan.
JMIR formative research
|May 17, 2024
概括
人工智能 (AI) 症状检查器在三年内没有改善诊断准确度. 对于不常见的疾病和非典型的表现,他们的表现较低,这凸显了需要更好的培训数据的需要.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 临床决策支持 临床决策支持
背景情况:
- 人工智能症状检查器需要真实世界的患者数据来提高诊断准确度.
- 对AI症状检查器性能的纵向评估是有限的.
- 人工智能症状检查器在临床实践中越来越多地使用.
研究的目的:
- 评估真实世界AI症状检查器诊断准确度的纵向变化.
- 为了评估AI症状检查器在三年内对差异诊断清单的准确性.
主要方法:
- 从2019年5月到2022年4月,对381名患者进行了回顾性观察性研究.
- 分析了人工智能生成的差异诊断列表与确认的最终诊断的准确性.
- 通过基平方测试,比较了三个不同的一年期的准确性.
主要成果:
- 整体AI症状检查器准确率为45.1% (172/381),在三年内没有显著改善 (P=.85).
- 对于不常见的疾病 (24.2%) 和非典型的表现 (14.5%),准确性明显较低.
- 常见的疾病和典型的表现与更高的AI精度 (P<.001) 有着强烈的关联.
结论:
- 人工智能症状检查器的诊断准确性在三年内并没有纵向改善.
- 不常见的疾病和非典型的表现为当前的AI症状检查者带来了重大挑战.
- 未来的AI症状检查器开发应该专注于培训具有多样化和不常见的条件.
相关概念视频
Receiver Operating Characteristic Plot
140
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
140
Diagnostic and Statistical Manual of Mental Disorders (DSM)
54
The Diagnostic and Statistical Manual of Mental Disorders (DSM) serves as the primary classification system for mental health disorders, providing standardized diagnostic criteria for clinicians and researchers. First published by the American Psychiatric Association (APA) in 1952, the DSM has undergone several revisions to reflect evolving psychiatric understanding. The fifth edition, DSM-5, released in 2013, introduced key updates that expanded diagnostic categories and modified diagnostic...
54
Sensitivity, Specificity, and Predicted Value
291
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
291


