Adaptive sample selection for individual test-sample prediction under distribution shift via minimum regularized...

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Distribution shift significantly degrades model performance when training and real-world data differ.
• Partial Least Squares (PLS) regression, common for high-dimensional data, is susceptible to distribution shift.
• Existing methods struggle to adapt PLS models to evolving data distributions.

Purpose of the Study:

• To introduce a novel method for improving PLS model accuracy under distribution shift.
• To enhance the generalization ability of PLS models by adaptively selecting training samples.
• To ensure numerical stability of covariance matrices in high-dimensional settings.

Main Methods:

• Developed the adaptive minimum regularized covariance determinant (AMRCD) method.
• AMRCD adaptively selects training samples matching the distribution of a single test sample.
• Incorporated regularization techniques to maintain well-conditioned covariance matrices.

Main Results:

• The AMRCD method significantly improved prediction accuracy for test samples.
• Outperformed classical PLS and an alternative sample selection framework.
• Validation conducted on three simulation and two real-world datasets.

Conclusions:

• AMRCD effectively addresses distribution shift challenges in PLS regression.
• The method enhances predictive accuracy and model generalization.
• AMRCD provides a robust solution for high-dimensional data analysis with varying distributions.