Related Experiment Video
Updated: Jul 11, 2026
09:00
Advancing Dyslexia Assessment in Children Through Computerized Testing
Published on: August 16, 2024
A modified sign test for comparing paired ROC curves
Thomas M Braun1, Todd A Alonzo
1Department of Biostatistics, University of Michigan, Ann Arbor, MI 48109, USA. tombraun@umich.edu
Biostatistics (Oxford, England)
|October 11, 2007
Summary
We developed a new permutation test to compare areas under the curve (AUCs) for paired data. This novel method is valid across different measurement scales and shows equal power to existing tests.
Area of Science:
- Biostatistics
- Statistical Methods
Background:
- Comparing diagnostic accuracy often involves paired data from diseased and non-diseased subjects.
- Assessing differences in areas under the curve (AUCs) is crucial for evaluating diagnostic tests.
Purpose of the Study:
- To introduce a novel permutation test for comparing AUCs in a paired setting.
- To ensure the test's validity across different measurement scales for paired modalities.
Main Methods:
- Developed a permutation test by shuffling diseased/non-diseased labels within each modality.
- The method is a sign test for the symmetry of an underlying discrete distribution.
- Validated through simulation studies.
Main Results:
- The proposed permutation test is valid even when modalities have different scales.
- Demonstrated operating characteristics and power through simulations.
- The test exhibits equal statistical power compared to the Bandos et al. (2005) permutation test.
Conclusions:
- The novel permutation test provides a valid approach for comparing AUCs in paired data.
- This method is robust to different measurement scales, enhancing its applicability.
- Offers comparable power to existing methods, serving as a reliable alternative.
More Related Videos
Related Concept Videos
01:15Receiver Operating Characteristic Plot
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
01:17Sign Test for Matched Pairs
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in value between...
To conduct the sign test, we first calculate the differences in value between...
01:10Introduction to the Sign Test
The sign test is an important tool in nonparametric statistics, offering a straightforward yet effective method for analyzing matched pairs, nominal data, or hypotheses concerning the median of a population. It transforms data points into positive or negative signs, avoiding the need for assumptions about data distribution and instead focusing on the direction of change. It is particularly valuable when data does not conform to the normal distribution requirements of many parametric tests. For...
01:19The Mantel-Cox Log-Rank Test
The Mantel-Cox log-rank test is a widely used statistical method for comparing the survival distributions of two groups. It tests whether a statistically significant difference exists in survival times between the groups without assuming a specific distribution for the survival data, making it a non-parametric test. This flexibility makes the log-rank test particularly valuable in medical research and other fields where the timing of an event, such as death or disease recurrence, is of interest.
01:09Wilcoxon Signed-Ranks Test for Matched Pairs
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
01:20Region of Convergence of Laplace Tarnsform
The Region of Convergence (ROC) is a fundamental concept in signal processing and system analysis, particularly associated with the Laplace transform. The ROC represents an area in the complex plane where the Laplace transform of a given signal converges, determining the transform's applicability and utility.
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This substitution...
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This substitution...