Breast cancer diagnosis from proteomic mass spectrometry data: a comparative evaluation

The performance results of a wide range of different classifiers applied to proteomic mass spectra data, in a blind comparative assessment organised by Bart Mertens, are reviewed. The different approaches are summarised, issues of how to evaluate and compare the predictions are described, and the results of the different methods are examined. Although the different methods perform differently, their rank ordering varies according to how one measures performance, so that one cannot draw unequivocal conclusions about which is 'best.' Instead, it is clear that what matters is not the method by itself, but the interaction of method and user - the degree of sophistication of the user with a method. Nevertheless, such competitions do serve the useful role of setting (constantly improving) baselines against which new researchers can pit their wits and methods, as well as providing standards against which new methods should be assessed.