The appropriateness evaluation protocol: a technique for assessing unnecessary days of hospital care

A major national health policy objective is to improve the efficiency of hospital utilization. To evaluate programmatic interventions with this objective, such as the Professional Standards Review Organization program, measures of appropriate use are a fundamental need. This report represents the results of two developmental trials of a new technique, labeled the Appropriateness Evaluation Protocol (AEP), for assessing potentially unnecessary hospital days of care. Twenty-seven objective criteria items related to medical services, nursing/life support services and patient condition factors were incorporated in the protocol. If any one of the criteria was met, the day was deemed "appropriate," and if none was met, the day was deemed "inappropriate" at an acute hospital level of care. A reviewer could override the objective criteria in either direction in reaching a final judgment. Three reviewers, two nurses and one physician each reviewed 200 charts at a teaching hospital. After correcting for a small number of abstracting errors, overall agreement rates between pairs of reviewers ranged from 92 to 94 per cent, levels significant p less than 0.0001. Of all cases judged inappropriate by at least one of the reviewers, specific agreement rates for the reviewer pairs on which days were inappropriate ranged from 73 to 79 per cent. These overall agreement rates and specific agreement rates on days of care judged as inappropriate are higher than those of any previously reported assessment methods. A parallel study of the appropriateness of admissions in these same cases, using purely subjective reviewer judgments, found overall agreement rates averaging 90 per cent, but rates of specific agreement on inappropriate admissions were less than 40 per cent between pairs of reviewers. Along with comparisons to other, more subjective, assessment techniques, this finding suggests that objective criteria are a vital element in developing methodologically sound techniques for assessing appropriate hospital use.