Health services research evaluation principles. Broadening a general framework for evaluating health information...

Area of Science:

• Health Informatics
• Health Services Research
• Information Systems Evaluation

Background:

• Existing health information technology (HIT) evaluations lack comprehensive frameworks, limiting generalizability and lessons learned.
• The American Recovery and Reinvestment Act of 2009 necessitates robust HIT evaluation methods.
• Current HIT evaluation studies often lack depth and breadth.

Purpose of the Study:

• To develop an informatics evaluation framework for HIT.
• Integrate health services research (HSR) and informatics evaluation components.
• Address shortcomings in existing HIT evaluation frameworks.

Main Methods:

• Systematic literature review updating Ammenwerth and deKeizer's review.
• Eliciting and organizing evaluation criteria into a framework.
• Applying the Health Information Technology Research-based Evaluation Framework (HITREF) to an electronic health record (EHR) evaluation.

Main Results:

• Identified 128 EHR evaluation studies and seven new criteria (e.g., Patient Privacy, Unintended Consequences, Patient Satisfaction).
• HITREF successfully guided survey construction, data analysis, and interpretation.
• The framework proved comprehensive, encompassing all emergent themes.

Conclusions:

• Recommend adopting a research-based HIT evaluation framework like HITREF.
• HITREF can enhance the monitoring of HIT challenges during increased adoption.
• Future EHR evaluators should consider integrating HITREF into their toolkits.