Systematic update of computerized physician order entry order sets to improve quality of care: a case study

Seattle Children's Hospital was one of the early adopters of computerized physician order entry. As part of our 2003 go-live, order sets were created opportunistically by using an ad hoc development process. A pilot study revealed that this ad hoc development process resulted in order sets that were neither internally nor externally consistent. We sought to update order sets by using software development techniques, to try and improve consistency and also to review clinical content so that they could be updated to current evidence and consensus-based best practice. We also sought to identify and categorize errors found in the original order sets.

This is a case study of a new order set development process that: (1) assigned order set ownership; (2) created and applied standards for how orders should appear and be organized within order sets; (3) supported multidisciplinary review and update; and (4) enforced submitting completed specifications before order set build. We extracted order sets into Microsoft Word specifications, updated content by using the Track Changes function, and then updated our Clinical Information System. Changes were reviewed and organized according to themes.

We created standard order formats for 98 orders; 191 order sets were standardized. Multidisciplinary review identified medication issues in 37% of order sets (used in 47.6% of inpatient admissions).

This case study demonstrates that it is not sufficient to simply implement computerized physician order entry. Clinical decision supports should be subject to rigorous development processes to ensure both clinical appropriateness and correctness.