Boosting K-nearest neighbor regression performance for longitudinal data through a novel learning approach.

Area of Science:

• Statistics and Data Science
• Biostatistics
• Machine Learning

Background:

• Longitudinal studies necessitate flexible prediction methods for response trajectories.
• Time-dependent and time-independent covariates pose challenges in longitudinal data analysis.
• Existing K-Nearest Neighbor (KNN) regression may struggle with large longitudinal datasets.

Purpose of the Study:

• To introduce Clustering-based KNN Regression for Longitudinal Data (CKNNRLD), a novel extension of KNN.
• To enhance prediction accuracy and computational efficiency for longitudinal data.
• To provide a robust tool for analyzing complex longitudinal datasets.

Main Methods:

• Data clustering using the K-means for longitudinal data (KML) algorithm.
• Nearest neighbor search restricted to relevant data clusters.
• Theoretical framework development and validation through extensive simulations and a real spirometry dataset.

Main Results:

• CKNNRLD demonstrates superior prediction accuracy compared to standard KNN.
• CKNNRLD significantly reduces execution time and computational burden.
• CKNNRLD was approximately 3.7 times faster than standard KNN for N=2000, with notable speed improvements for N > 100 and N > 500.

Conclusions:

• CKNNRLD offers substantial improvements in accuracy and computational efficiency over traditional KNN.
• The algorithm is particularly beneficial for researchers managing large longitudinal datasets.
• CKNNRLD presents a valuable advancement for longitudinal data prediction.