AUCReshaping: improved sensitivity at high-specificity.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep learning (DL) model evaluation often uses Area Under the Receiver-Operating-Curve (AU-ROC), which may not reflect performance at critical operating points.
• Class imbalance in anomaly detection datasets poses challenges, leading to potential misclassification costs, especially for abnormality detection.

Purpose of the Study:

• To introduce AUCReshaping, a novel technique to optimize DL model performance within specific sensitivity and specificity ranges.
• To address the limitations of traditional AU-ROC evaluation in anomaly detection tasks.

Main Methods:

• AUCReshaping modifies the Receiver-Operating-Curve (ROC) by optimizing sensitivity at a predetermined specificity level.
• An adaptive, iterative boosting mechanism is employed to focus the network on relevant samples during training.
• The technique was evaluated on Chest X-Ray (CXR) analysis, breast mammogram analysis, and credit card fraud detection.

Main Results:

• AUCReshaping demonstrated substantial improvements in sensitivity, ranging from 2% to 40%, at high specificity levels for binary classification tasks.
• The method effectively reshapes the ROC curve within desired operational ranges, enhancing model utility.

Conclusions:

• AUCReshaping offers a significant advancement for deep learning-based anomaly detection, particularly in domains requiring high specificity.
• The technique provides a more targeted performance optimization than traditional AU-ROC evaluation for critical applications.