Sequential label shift detection in classification data: An application to dengue fever.

Area of Science:

• Epidemiology
• Machine Learning
• Biostatistics

Background:

• Dengue fever diagnosis relies on classifiers assuming stable population prevalence.
• Changes in dengue prevalence, like during outbreaks, necessitate rapid detection for public health action and model recalibration.

Purpose of the Study:

• To develop a method for detecting distribution changes in unlabeled, sequentially observed classification data.
• To specifically address label shift, where class priors change but class conditional distributions remain constant.

Main Methods:

• The problem is reframed as detecting changes in one-dimensional classifier scores.
• Nonparametric sequential changepoint detection procedures are employed.
• Classifier training data is utilized to estimate the detection statistic.

Main Results:

• The proposed method effectively detects label shift in classification data.
• Performance is validated using simulated dengue outbreaks with real-world data.
• The approach demonstrates superior performance compared to existing detection procedures in label shift scenarios.

Conclusions:

• The developed changepoint detection procedures are effective for identifying shifts in dengue prevalence.
• This method offers a valuable tool for early outbreak detection and adaptive diagnostic model management.
• The nonparametric approach is robust and converges to parametric counterparts with sufficient training data.