Related Experiment Videos
When and how to evaluate health information systems?
Jeremy C Wyatt1, Sylvia M Wyatt
1Knowledge Management Centre, University College London, Gower Street, London WC1E 6BT, UK. jwyatt@nice.nhs.uk
International Journal of Medical Informatics
|June 18, 2003
Summary
Evaluating health information systems (HIS) requires careful method selection. The best approach depends on the specific questions and desired reliability, not the technology itself.
Area of Science:
- Health Informatics
- Health Services Research
- Information Systems Evaluation
Background:
- Evaluating large-scale health information systems (HIS) presents significant challenges.
- There is a need for clear guidance on appropriate evaluation methodologies for policymakers and system implementers.
Purpose of the Study:
- To outline the necessity of evaluating HIS.
- To present a range of suitable evaluation methods in accessible language.
- To provide resources for further information on HIS evaluation.
Main Methods:
- Discussion of qualitative and quantitative research methods, including before-after, controlled before-after, and randomized designs.
- Illustrative example of evaluating an order communications system's impact on lab requests.
- Description of biases affecting impact studies and strategies for mitigation.
Main Results:
- Identification of common biases in health information system impact studies.
- Comparison of trade-offs between simple, less reliable studies and complex, rigorous trials.
- Emphasis on the cost-benefit analysis of different evaluation designs.
Conclusions:
- The choice of evaluation method is dictated by the research questions and required reliability, not the specific technology (e.g., HIS or pharmaceuticals).
- Decision-making authority for selecting evaluation studies rests with those commissioning them.
- Effective HIS evaluation ensures reliable answers to critical questions regarding system impact and effectiveness.