Related Experiment Video
Updated: Sep 20, 2026

Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Applying methodology to electronic medical record selection
Anne Holbrook1, Karim Keshavjee, Sue Troyan
1Centre for Evaluation of Medicines, St. Joseph's Hospital, 105 Main Street East, Level P1, Hamilton, Ont., Canada L8N 1G6. holbrook@mcmaster.ca
Background:
Given the potential for electronic medical records (EMRs) to influence every aspect of health care, there has been surprisingly little rigorous research applied to this emerging health technology. An initial phase of the COMPETE (computerization of medical practices for the enhancement of therapeutic efficacy) program, which investigates the impact of EMRs and electronic decision support on efficiency, quality of care and privacy concerns, developed and used a rigorous approach to EMR selection.
Methods:
A multidisciplinary team with clinical, technical and research expertise led an eight-stage evaluation process with direct input from potential clinical users at each stage. An iterative sequence of review of EMR specifications and features, live product demonstrations, site visits, and negotiations with vendors led to a progressive narrowing of the field of eligible EMR systems. Final scoring was based on three main themes of clinical usability, data quality and vendor maturity issues.
Results:
The field of eligible EMR systems was relatively easily narrowed to a few finalists. Determination of the important strengths and weaknesses of these systems' usability, data extraction capabilities and vendor viability, required the full suite of evaluation steps. Preferences varied amongst clinicians and between clinicians and technical specialists, particularly regarding the importance of user interface versus database issues. However, the final scoring process showed consensus amongst clinical and technical experts.
Conclusion:
A rigorous, multidisciplinary process is useful in EMR selection. While prolonged and laborious, it is transparent, enhances buy in and realism of expectations by multiple potential users and IT support staff and may, therefore, improve the ultimate success of any EMR implementation project.
