Understanding prognostic models: The example of the PHASES score for unruptured intracranial aneurysms

Prognostic studies may inform individuals about the future course of their disease and help clinical decision making, but problems abound.

We summarize a study on the natural history of unruptured intracranial aneurysms (UIAs) and review the various steps in the construction of prognostic models. We emphasize the fundamental inductive problems of prognostic studies that attempt to use the backward road from the extension of patients suffering an event to create a new intensional definition of classes of patients at risk.

The first step in a prognostic model is to identify candidate baseline variables to be entered into the model, according to background knowledge, previous studies, and statistical associations with the event of interest. This is a multivariate task. The modeler already knows the outcomes the model is supposed to 'predict', so that multiple models are tested against the data until a satisfactory fit is obtained. The variables used to construct the model should not be added in an ad hoc fashion to fit heterogeneous studies. They should be selected in such a fashion as to be exportable outside the study to new patients. An infinite number of models can fit the same data. Thus, the most important step is to validate the prognostic value of the model in patients that were not used to construct the model. In the case of UIAs, this has never been done.

Prognostic studies present multiple problems. Unvalidated models should not be used in clinical practice.