Related Experiment Videos
Evaluation of a cardiac diagnostic program in a typical clinical setting
Hamish S F Fraser1, William J Long, Shapur Naimi
1Informatics Group, Children's Hospital, Boston, MA, USA. hamish@medg.lcs.mit.edu
Context:
The Heart Disease Program (HDP) is a novel computerized diagnosis program incorporating a computer model of cardiovascular physiology. Physicians can enter standard clinical data and receive a differential diagnosis with explanations.
Objective:
To evaluate the diagnostic performance of the HDP and its usability by physicians in a typical clinical setting.
Design:
A prospective observational study of the HDP in use by physicians in departments of medicine and cardiology of a teaching hospital. Data came from 114 patients with a broad range of cardiac disorders, entered by six physicians.
Measurements:
Sensitivity, specificity, and positive predictive value (PPV). Comprehensiveness: the proportion of final diagnoses suggested by the HDP or physicians for each case.
Relevance:
the proportion of HDP or physicians' diagnoses that are correct. Area under the receiver operating characteristic (ROC) curve (AUC) for the HDP and the physicians. Performance was compared with a final diagnosis based on follow-up and further investigations.
Results:
Compared with the final diagnoses, the HDP had a higher sensitivity (53.0% vs. 34.8%) and significantly higher comprehensiveness (57.2% vs. 39.5%, p < 0.0001) than the physicians. Physicians' PPV and relevance (56.2%, 56.0%) were higher than the HDP (25.4%, 28.1%). Combining the diagnoses of the physicians and the HDPs, sensitivity was 61.3% and comprehensiveness was 65.7%. These findings were significant in the two collection cohorts and for subanalysis of the most serious diagnoses. The AUCs were similar for the HDP and the physicians.
Conclusions:
The heart disease program has the potential to improve the differential diagnoses of physicians in a typical clinical setting.