Foundations of Feature Selection in Clinical Prediction Modeling

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Biostatistics

Background:

• Feature selection is crucial for developing parsimonious yet predictive clinical models.
• Balancing model complexity with predictive performance is a key challenge.
• Clinical utility necessitates considering data availability and computational resources.

Purpose of the Study:

• To elucidate the importance and pitfalls of feature selection in clinical prediction modeling.
• To review and demonstrate various feature selection methodologies.
• To provide guidance on selecting appropriate features for clinical models.

Main Methods:

• Demonstration of simple filtering methods: correlation, significance, and variable importance.
• Explanation of intrinsic feature selection methods: Lasso, tree-based, and rule-based approaches.
• Focus on two algorithmic wrapper methods: Recursive Feature Elimination (RFE) and Purposeful Variable Selection (Hosmer and Lemeshow).

Main Results:

• Simple filtering methods offer basic feature reduction.
• Intrinsic methods like Lasso provide embedded feature selection.
• Wrapper methods, particularly RFE, offer robust feature selection applicable across model types.

Conclusions:

• Effective feature selection is vital for robust clinical prediction models.
• A range of methods exist, from simple filters to complex wrappers like RFE.
• The choice of method depends on clinical context, data, and desired model characteristics.