Intelligent computer reporting 'lack of experience': a confidence measure for decision support systems

The purpose of this study was to explore the feasibility of developing artificial neural networks that are able to provide confidence measures for their diagnostic advice. Computer-aided decision making can improve physician performance, but many physicians hesitate to use these 'black boxes'. If we are to rely upon decision support systems for such tasks as medical diagnosis it is essential that the computers indicate when the advice given is based on experience, i.e. give a confidence measure. An artificial neural network was trained to diagnose healed anterior myocardial infarction and to indicate 'lack of experience' when test electrocardiograms were different from the electrocardiograms of the training set. A database of 1249 electrocardiograms from patients who had undergone cardiac catheterization was used to train and test the neural network. Thereafter, the ability of the network to indicate 'lack of experience' was assessed using 100 left bundle branch block electrocardiograms, an electrocardiographic pattern that was excluded from the training set. The network indicated that 83% of the left bundle branch block electrocardiograms and 1% of the test electrocardiograms from catheterized patients were different from the electrocardiograms of the training set. All but one of the left bundle branch block electrocardiograms would otherwise be falsely classified as anterior myocardial infarction by the network. Artificial neural networks can be trained to indicate 'lack of experience', and this ability increases the possibility for neural networks to be accepted as reliable decision support systems in clinical practice.