Broadening our approach to evaluating medical information systems

Evaluation in medical informatics tends to follow the paradigm of controlled clinical trials. This model carries with it a number of assumptions whose implications for medical informatics deserve examination. In this paper, we describe the conventional wisdom on evaluation, pointing out some of its underlying assumptions and suggesting that these assumptions are problematic when applied to some aspects of evaluation. In particular, we believe that these assumptions contribute to the problem of user acceptance. We then suggest a broader approach to evaluation, offering some conceptual and methodological distinctions that we believe will be of use to the medical informatics community in rethinking this issue.