How valid is the AHRQ Patient Safety Indicator "postoperative respiratory failure"?

The Agency for Healthcare Research and Quality Patient Safety Indicator postoperative respiratory failure (PRF) uses administrative data to screen for potentially preventable respiratory failure after elective surgery based on a respiratory failure diagnosis or an intubation or ventilation procedure code. Data on PRF accuracy in identifying true events is scant; a recent study using University HealthSystem Consortium data found a positive predictive value (PPV) of 83%. We examined the indicator's PPV in the Veterans Health Administration.

We applied the Patient Safety Indicator software (v.3.1a) to fiscal year 2003-2007 VA discharge data. Trained abstractors reviewed medical records of 112 software-flagged PRF cases. We calculated the PPV and examined false positives to determine reasons for incorrect identification and true positives to determine clinical consequences and potential risk factors of PRF.

Seventy-five cases were true positive (PPV 67%; 95% CI, 57-76%); 13% were identified by a diagnosis code, 53% by a procedure code, 33% by both. Of false positives, 19% represented coding errors, 76% represented nonelective admissions. Of true positives, 28% of patients died, 56% had an American Society of Anesthesiologists level higher than II. Of associated index procedures, 53% were abdominal/pelvic, and 56% lasted >3 hours.

Based on our and University HealthSystem Consortium's findings, PRF should continue to be used as a screen for potential patient-safety events. Its PPV could be substantially improved in the Veterans Health Administration through introduction of an admission status code. Many PRF-identified cases appeared to be at high risk, based on patient and procedure-related factors. The degree to which such cases are truly preventable events requires additional assessment.