Developing criteria for ordering common ancillary services

This article describes the process utilized to develop criteria for appropriately ordering five ancillary services frequently performed in hospitals: 1) arterial blood gases, 2) electrocardiograms (ECGs), 3) serum electrolytes, 4) chest x-rays, and 5) complete blood counts (CBCs). The development of each set of criteria involved an initial consultant, a panel of six additional regional expert consultants, and three national reviewers. Each criteria set was developed through a process involving seven steps: 1) an initial working draft, 2) revisions at an initial meeting of the regional experts, 3) revisions at a second meeting of the regional experts, 4) written comments from the regional experts, 5) written comments from national reviewers, 6) additional written comments from regional experts, and 7) application of the criteria to cases in a community hospital. The change in item content was measured between steps and agreement with individual items was measured at steps 2, 3, and 4. The results indicate that appreciable change in content occurred with each step except step 4. Agreement started fairly high and was over 90% by step 4. The discussion considers the utility of each developmental step, factors affecting the utility of the criteria, the extent to which the results can be generalized, and the need for more research to identify optimal processes for the development of criteria to evaluate quality of care.