Evaluation of health information applications--challenges ahead of us

The aim of the paper is to review the challenges for evaluation in the light of characteristics of the healthcare sector, present as well as future.

The approach is a synthesis based on highlights from the literature.

The review addresses the following issues: 1) the role of evaluation activities within a systems development or implementation context; 2) suggestions on the nature of success and failure characteristics; and 3) evaluation aspects viewed in the perspective of different types of systems. Constructive evaluation, evaluation being the act of bringing about a decision-making basis, is perceived as the means to minimize failure and maximize success from the very beginning of the development or implementation. Based on these discussions, the challenges that evaluation and evaluators are facing are debated.

The ultimate challenge ahead is first to fill the gap of presently needed evaluation methods. This need is in particular related to evaluation of cognitive and work process-oriented aspects of IT-based solutions. Finally, the challenge is to provide constructive evaluation methods and methodologies for dealing with the full complexity and dynamics of the target domain, for application within the entire lifecycle of the IT-based systems and solutions.