涉及数千名竞争对手的公开竞赛未能为新的诊断测试准确性系统审查构建有用的抽象分类器
Yuki Kataoka1,2,3,4, Shunsuke Taito2,5, Norio Yamamoto2,6,7
1Department of Internal Medicine, Kyoto Min-iren Asukai Hospital, Kyoto, Japan.
Research synthesis methods
|June 20, 2023
概括
开发用于诊断测试准确性 (DTA) 系统审查的机器学习抽象分类器是具有挑战性的. 竞争没有产生一个具有足够回忆力的分类器,可以立即用于DTA审查.
科学领域:
- 医疗信息学 医疗信息学
- 系统审查方法论 系统审查方法论
背景情况:
- 目前,针对诊断测试准确性的系统审查 (DTA) 缺乏自动抽象分类器.
- 从数据库搜索中有效选初级DTA研究摘要是一个重大挑战.
研究的目的:
- 开发基于机器学习的抽象分类器,用于新的DTA系统审查.
- 评估使用公开竞争来创建这些分类器的可行性.
主要方法:
- 编制了来自11个DTA系统审查的摘要数据集.
- 为了让参与者开发和提交抽象分类器,举行了公开竞赛.
- 分类器在训练集上接受培训,并使用公共和私人测试集进行验证,Fbeta (β=7) 作为评估指标.
主要成果:
- 在为期4个月的比赛中,来自1400多名参赛者的13000多份作品被收到.
- 在外部验证数据上,表现最好的分类器获得了0.4036的Fbeta得分和0.2352的回忆.
- 开发的分类器没有达到实际应用所需的召回值.
结论:
- 该研究无法开发一种机器学习的抽象分类器,适合在DTA系统审查中立即使用.
- 需要进一步的研究来提高分类器的性能,并在各种临床领域验证它们.
相关概念视频
Documentation of Nursing Diagnosis
1.3K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.3K
Receiver Operating Characteristic Plot
285
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
285
Sensitivity, Specificity, and Predicted Value
513
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
513
Accuracy and Errors in Hypothesis Testing
229
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
229
Survival Tree
118
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
118


