Prudent expert systems with credentials: managing the expertise of decision support systems

'Black box' expert systems (ES) are mistrusted by clinicians. Errors generated by medical ES are also a significant cause for concern. We report new ES properties--prudence and credentials--that improve error management and underpin a new approach for improving the credibility of ES for clinical users. Prudent ES modify their output according to past experience. For a knowledge base built from 1610 cases, feature exception prudence (FEP) detected all interpretation errors (100% sensitivity for error detection). Although the false positive rate for FEP was high (47%), the 100% sensitivity meant that the 53% of cases that did not produce flags could be exempted from human validation. As more cases are processed, fewer cases should need human validation. Feature recognition prudence (FRP), a property of ripple down rules (RDR), proposed the correct alternative conclusion in 14% of incorrectly interpreted cases. Human expert validation of the flagged cases enabled context-sensitive credentials (accuracy, incidence and specificity of a given conclusion) to accumulate. Credentials should enable the user to judge the credibility of the ES output. An error management strategy based on credentialed, prudent ES should reduce the impact of error in the clinical environment. The empowerment of clinicians to critically evaluate ES credibility may facilitate greater confidence in, and acceptance of, ES by clinicians.