Nested logistic regression models and ΔAUC applications: Change-point analysis

Area of Science:

• Statistics
• Biostatistics
• Machine Learning

Background:

• The area under the receiver operating characteristic curve (AUC) is a key metric for predictive model performance.
• Change in AUC (ΔAUC) assesses added predictors' value in nested models.
• Existing methods for ΔAUC are limited by non-normal distributions under certain conditions.

Purpose of the Study:

• To review ΔAUC usage for nested logistic models and its distributional challenges.
• To propose a new statistical test for nested logistic models with a change-point predictor.
• To develop a resampling scheme for critical values and bootstrap inference for change-point parameters.

Main Methods:

• Review of ΔAUC for nested logistic models.
• Development of a novel test statistic for change-point models based on ΔAUC.
• Implementation of a resampling scheme for critical value approximation.
• Application of m-out-of-n bootstrap for change-point parameter inference.

Main Results:

• Demonstration of ΔAUC's degeneracy and non-normal distribution issues.
• Proposal of a new ΔAUC-based test statistic for change-point models.
• Validation of the proposed method through large-scale simulations.
• Application to real-life datasets for practical illustration.

Conclusions:

• The proposed ΔAUC test effectively handles nested logistic models with change-point predictors.
• The resampling scheme and bootstrap inference provide reliable critical values and parameter estimates.
• The method offers a robust approach for evaluating predictive accuracy improvements in complex models.