Comparing variable and feature selection strategies for prediction - protocol of a simulation study in...

Area of Science:

• Biostatistics
• Machine Learning
• Clinical Informatics

Background:

• Machine learning (ML) is increasingly used for clinical prediction models, often outperforming traditional methods.
• Variable selection is crucial for developing accurate and interpretable prediction models, especially in low-dimensional settings.

Purpose of the Study:

• To compare the performance of different variable selection methodologies in clinical prediction models.
• To evaluate variable selection strategies based on predictive accuracy, variability, and descriptive accuracy.
• To provide a protocol for a simulation-based analysis of variable selection techniques.

Main Methods:

• A simulation study design using the Aims, Data, Estimands, Methods, and Performance (ADEMP) framework.
• Comparison of six distinct statistical learning approaches for both data generation and model learning.
• Analysis of low-dimensional datasets encompassing classical and machine learning paradigms.

Main Results:

• The study protocol outlines the planned simulation-based comparison of variable selection strategies.
• Key performance metrics include relative predictive accuracy, its variability, and descriptive accuracy.
• Six statistical learning methods will be assessed across various simulation scenarios.

Conclusions:

• This protocol details a rigorous simulation study to compare variable selection methods in clinical prediction.
• Findings will inform the optimal choice of variable selection techniques for low-dimensional clinical datasets.
• The study aims to enhance the reliability and accuracy of machine learning-based clinical prediction models.