Statistical assessment of change point detectors for single molecule kinetic analysis

Change point detectors (CPDs) are used to segment recordings of single molecules for the purpose of kinetic analysis. The assessment of the accuracy of CPD algorithms has usually been based on testing them with simulated data. However, there have not been methods to assess the output of CPDs from real data independent of simulation. Here we present one method to do this based on the assumption that the elementary kinetic unit is a stationary period (SP) with a normal distribution of samples, separated from other SPs by change points (CPs). Statistical metrics of normality can then be used to assess SPs detected by a CPD algorithm (detected SPs, DSPs). Two statistics in particular were found to be useful, the z-transformed skew (S(Z)) and z-transformed kurtosis (K(Z)). K(Z)(S(Z)) plots of DSP from noise, simulated data and single ion channel recordings showed that DSPs with false negative CP could be distinguished. Also they showed that filtering had a significant effect on the normality of data and so filtering should be taken into account when calculating statistics. This method should be useful for analyzing single molecule recordings where there is no simple model for the data.