Use of severity of illness to evaluate quality of care

This study compared a severity of illness system (APACHE II) and a 10% random sample of charts in terms of their ability to identify cases with quality problems. Using condition-specific data bases of 337 pneumonia, 363 acute myocardial infarction and 266 hip fracture charts, severity of illness information was used to separate cases into those with a high and a low likelihood of a poor outcome. Cases with low admission severity of illness combined with subsequent death were flagged as potential quality problems. Physician evaluation was used as the gold standard to measure flag performance. Flags were tested against a 10% random sample drawn from within the three condition-specific data bases. Analyses focused on a combination of sensitivity and positive predictive value. The low severity plus death flag performed much better than a 10% random sample approach, suggesting that outcomes monitoring flags based on severity of illness could play an important role in screening cases for potential quality problems.