Entering the black box of neural networks

Artificial neural networks have proved to be accurate predictive instruments in several medical domains, but have been criticized for failing to specify the information upon which their predictions are based. We used methods of relevance analysis and sensitivity analysis to determine the most important predictor variables for a validated neural network for community-acquired pneumonia.

We studied a feed-forward, back-propagation neural network trained to predict pneumonia among patients presenting to an emergency department with fever or respiratory complaints. We used the methods of full retraining, weight elimination, constant substitution, linear substitution, and data permutation to identify a consensus set of important demographic, symptom, sign, and comorbidity predictors that influenced network output for pneumonia. We compared predictors identified by these methods to those identified by a weight propagation analysis based on the matrices of the network, and by logistic regression.

Predictors identified by these methods were clinically plausible, and were concordant with those identified by weight analysis, and by logistic regression using the same data. The methods were highly correlated in network error, and led to variable sets with errors below bootstrap 95% confidence intervals for networks with similar numbers of inputs. Scores for variable relevance tended to be higher with methods that precluded network retraining (weight elimination) or that permuted variable values (data permutation), compared with methods that permitted retraining (full retraining) or that approximated its effects (constant and linear substitution).

Methods of relevance analysis and sensitivity analysis are useful for identifying important predictor variables used by artificial neural networks.