Influence Analysis for the Area Under the Receiver Operating Characteristic Curve

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Analysis

Background:

• Classification measures are crucial for evaluating and building predictive models.
• Individual data points can disproportionately influence these performance metrics, compromising reliability.
• The area under the receiver operating characteristic curve (AUC) is a widely used and important classification measure.

Purpose of the Study:

• To develop methods for identifying influential observations that impact AUC estimates.
• To enhance the robustness and reliability of classifier performance assessments.

Main Methods:

• Proposed novel indexes based on influence functions and local influence concepts.
• Utilized cumulative lift charts to reconcile disagreements among proposed indexes.
• Methods rely solely on classification scores and are applicable to various classifiers.

Main Results:

• Successfully developed and illustrated indexes to detect influential observations affecting AUC.
• Demonstrated the utility of cumulative lift charts in conjunction with proposed indexes.
• Confirmed applicability to classifiers generating real-valued scores.

Conclusions:

• The proposed influence-based indexes effectively identify observations impacting AUC.
• Graphical tools complement the indexes, offering a robust approach to outlier detection.
• These methods improve the reliability of classifier evaluation in practical applications.