Related Experiment Videos
The sampling distributions of Gaussian ROC statistics
Neil A Macmillan1, Caren M Rotello, Jeff O Miller
1Department of Psychology, University of Massachusetts, Amherst, Massachusetts 01003, USA. nam@psych.umass.edu
Perception & Psychophysics
|July 31, 2004
Summary
This study analyzes the Receiver Operating Characteristic (ROC) curve model for discrimination experiments. Statistical bias was minimal for area under the ROC (Az) but substantial for the slope (s), impacting ROC analysis.
Area of Science:
- Psychology
- Psychophysics
- Signal Detection Theory
Background:
- Receiver Operating Characteristic (ROC) curves are crucial for summarizing discrimination performance across confidence levels.
- The normal-distribution model often accurately describes ROCs, showing a linear relationship between z-transformed hit and false-alarm rates.
Purpose of the Study:
- To investigate the sampling distributions of key ROC model parameters: area under the ROC (Az), normalized difference (da), and slope (s).
- To assess the statistical bias and variability associated with these parameters in a ratings procedure.
Main Methods:
- Analysis of sampling distributions for Az, da, and s derived from a normal-distribution ROC model.
- Evaluation of statistical bias and variability influenced by trial number, response categories, and sensitivity.
Main Results:
- Trivial statistical bias was observed for Az, while bias was small for da and substantial for s.
- Sampling distribution variability decreased with increased trials and was affected by response categories and sensitivity.
Conclusions:
- The study quantifies the statistical properties of ROC parameters, highlighting biases in slope estimation.
- Provides tools (figures and tables) for constructing confidence intervals and hypothesis testing for ROC statistics.