An efficient variance estimator of AUC and its applications to binary classification.

Area of Science:

• Statistics
• Machine Learning
• Medical Data Analysis

Background:

• Area under the ROC curve (AUC) is a key metric for binary classifier performance.
• Sampling variation necessitates assessing the reliability of AUC estimates.
• Current methods for AUC variance estimation can be computationally intensive.

Purpose of the Study:

• To develop an unbiased variance estimator for AUC estimates.
• To generalize the estimator for K-sample U-statistics.
• To provide a computationally efficient method for AUC variance estimation.

Main Methods:

• Extended Wang and Lindsay's proposal to a two-sample U-statistic form.
• Employed a partition-resampling scheme for computational efficiency.
• Validated the estimator through simulation studies and real-world medical datasets.

Main Results:

• The proposed AUC variance estimator is unbiased.
• It demonstrates comparable or superior performance to jackknife and bootstrap methods.
• Achieved computational times 10-30 times faster than existing methods.

Conclusions:

• The developed AUC variance estimator offers an efficient and accurate alternative.
• It is applicable for model selection (e.g., one-standard-error rule) and confidence interval construction.
• The method shows practical utility in medical sciences.