ROCS: receiver operating characteristic surface for class-skewed high-throughput data.

Area of Science:

• Bioinformatics
• Machine Learning
• Statistical Modeling

Background:

• Receiver Operating Characteristic (ROC) curves are standard for classifier performance evaluation.
• High-throughput data analysis often presents class-skewed datasets, where most features are true negatives.
• Traditional ROC curves and Area Under the Curve (AUC) are insufficient for class-skewed data, limiting performance assessment.

Purpose of the Study:

• To introduce a novel method for evaluating classifier performance on class-skewed high-throughput data.
• To define an ROC Surface (ROCS) that incorporates True Discovery Rate (TDR) alongside True Positive Rate (TPR) and False Positive Rate (FPR).
• To provide complementary metrics like Volume Under the Surface (VUS) and FDR-controlled AUC (FCAUC) for robust evaluation.

Main Methods:

• Development of the ROC Surface (ROCS) concept.
• Integration of True Discovery Rate (TDR) into the ROC analysis framework.
• Calculation of Volume Under the Surface (VUS) and FDR-controlled AUC (FCAUC).
• Implementation of the ROCS method in an R package.

Main Results:

• The ROC Surface (ROCS) provides a more comprehensive performance measure for imbalanced datasets.
• VUS and FCAUC offer valuable insights into classifier performance where traditional AUC falls short.
• The proposed method effectively addresses the limitations of ROC curves in high-throughput, class-skewed scenarios.

Conclusions:

• ROCS, VUS, and FCAUC offer a superior approach for evaluating classifiers on class-skewed high-throughput data.
• This method enhances the reliability of performance assessment in challenging biological and data science contexts.
• An R package is available for practical implementation of the ROCS methodology.