Signal detection in averaged evoked potentials: Monte Carlo comparison of the sensitivity of different methods

Many clinical and research applications rely on detecting evoked potential (EP) signal or EP differences between conditions. Statistical methods for objective signal detection should be sensitive to the presence of signal, but must provide the user strict control on tolerated false alarm rate. The respective sensitivities of 6 signal detection methods were compared through several Monte Carlo simulations involving 2 autocorrelation structures, 5% and 1% significance levels, 8, 10 or 12 replications per study, and increasing signal to noise ratio. The signal detection methods compared were: (1) the Record Orthogonality Test by Permutations (ROT-p), a variant of the Residual Orthogonality Test (Achim et al., 1988), that provides an unbiased estimate of the energy of the signal present in the averaged data, (2) the Tsum2 permutation test of Karniski et al. (1994), (3) a Principal Component Analysis method (PC1) consisting of a t test on the weights of the first principal component, (4) multiple t tests on amplitudes with empirical adjustment for global false alarm rate, and (5-6) the test of Guthrie and Buchwald (1991) on length of consecutive t tests significant at P < 0.05 or 0.01 per-test. The first 3 methods did not exceed their nominal false alarm rate and clearly outperformed the last 3, with the ROT-p method being significantly more sensitive than all others under almost all conditions.