A web-based tool for the assessment of discrimination and calibration properties of prognostic models

Prognostic models are developed to assist clinicians in making decisions regarding treatment and follow-up management. The accuracy of these models is often assessed either in terms of their discrimination performance or calibration but rarely both. In this paper, we describe the development of an online tool for discrimination using Harrell C index and calibration using a Hosmer-Lemeshow type analysis (http://clinengnhs.liv.ac.uk/AADP/AADP_Welcome.htm). We show examples of using the tool on real data. We highlight situations where the model performed well in terms of either discrimination or calibration but not both depending on the sample size of the test set. We conclude that prognostic models should be assessed both in terms of discrimination and calibration and that calibration analysis should be carried out numerically and graphically.