Validity of four models for predicting outcome in critically ill acute renal failure patients

Several mathematical models are used to predict the survival of patients in renal failure and have become standards. Few of these models have been sufficiently validated, however, and their applicability at other institutions has been established.

We attempted to validate four models used to predict mortality in the setting of acute renal failure (Lohr, Bullock, Cioffi, APACHE II) with a registry of 512 ICU patients who received acute dialysis at our institution between 1988 and 1992. Using mortality as an endpoint, we applied each model as described by the original authors and compared the predicted results against the actual outcomes recorded in the registry.

The Lohr Model is based on five risk factors thought to predict mortality or survival. Although we found these five factors were strongly associated with mortality (p < 0.001), more than 20% of the highest-risk group survived. The Bullock Model predicted only 20% of the mortality of our data. For the Cioffi Model, the original study found a clear discrimination score (survivors: 1.76, non-survivors: -0.423); however, we found no significant difference in the score discriminating survivors and non-survivors (survivors: 0.70, non-survivors: 0.71, p = 0.96). APACHE II is a risk model consisting of a premorbid assessment of chronic health and 12 physiologic variables taken during ICU admission. We applied APACHE II to both our cardiac and non-cardiac patients, and found that this score did not discriminate between survivors and non-survivors (p values ranged from 0.37 to 0.62) in acute renal failure in either case: the original APACHE II system was not applied to cardiac surgery patients.

We conclude that models with good performance in their institutions of origin may not be valid in other institutions. Before using a predictive model, validity testing at the specific institution should be undertaken.