Guidelines for assessing performance of ST analysers

This paper proposes principles and methods for assessing performance of ST analysers and algorithms. We describe an evaluation protocol and performance measures suitable for assessing the accuracy of: (1) detecting episodes of ischaemic ST changes, (2) distinguishing between ischaemic and non-ischaemic ST change episodes, and (3) measuring ST deviation and ischaemia duration. There is generally not a one-to-one correspondence between reference and analyser-annotated ST episodes, nor can non-events be counted. Sensitivity and positive predictivity measures which assess the accuracy of detecting ischaemic ST episodes and total ischaemic time are based on the concepts of matching and overlap, respectively. To address the question of predicting performance in a clinical environment, we have utilized the bootstrap statistical procedure, which estimates the mean as well as the standard deviation of the analyser's expected performance. We illustrate the use of the evaluation protocol and performance measures by a case study in which we present an evaluation of our 2-channel Karhunen-Loève transform based ST change detection algorithm using the European Society of Cardiology ST-T database.