Exploring Different Approaches in Measuring EHR-based Adherence to Best Practice - A Case Study with Order Sets and Associated Outcomes

In connection with a recent enterprise-wide rollout of a new electronic health record, Intermountain Healthcare is investing significant effort in building a central library of best-practice order sets. These order sets represent best practice guidelines for specific clinical scenarios and are deployed with the intent of standardizing care, reducing variation, and consistently delivering good clinical outcomes to the populations we serve. The importance of measuring their use and the level to which caregivers adhere to these standards becomes an important factor in understanding and characterizing the impact that they deliver. Notwithstanding the importance of these metrics, well- defined methods for measuring adherence to a given clinical guideline as delivered through an order set are not fully characterized in the medical literature. In this paper, we describe initial efforts at measuring compliance to a defined 'best practice' standard by means of content utilization analysis, a calculated adherence model, and relevant clinical key performance indicators. The degree to which specified clinical outcomes vary across these measurement models are compared for a group of order sets tied to treating coronary artery bypass graft patients and heart failure patients. While the patterns derived from this analysis show some uncertainty, more granular methods that look at line-item, or 'order level' detail reveal more significant differences in the corresponding set of outcomes than higher-level adherence surrogates.