Prediction using step-wise L1, L2 regularization and feature selection for small data sets with large number of...

Area of Science:

• Computational Biology
• Cheminformatics
• Machine Learning

Background:

• Machine learning (ML) is widely used for biological predictions, but limited training data and numerous molecular descriptors pose challenges.
• Developing accurate predictive models requires careful feature selection when dealing with complex biological data.

Purpose of the Study:

• To develop and evaluate a robust two-step ML method for biological regression tasks with limited training data and extensive features.
• To address the challenge of high-dimensional feature spaces in biological prediction problems.

Main Methods:

• A two-step feature selection and model training procedure was applied to four biological regression datasets.
• The first stage utilized L1 regularization for optimized feature selection, reducing thousands of descriptors to approximately 50.
• The second stage employed L2 regularization on selected features, coupled with a soft loss function to mitigate outlier influence.

Main Results:

• The method successfully reduced over 6,000 features to around 50 in the initial stage.
• The two-step approach demonstrated improved prediction performance across all four CoEPrA regression tasks.
• The use of L2 regularization in the second stage further enhanced predictive accuracy.

Conclusions:

• The proposed two-step ML method is effective for biological prediction problems with limited sample sizes and high-dimensional descriptor spaces.
• This approach offers a valuable tool for drug discovery and other areas requiring accurate biological predictions from complex datasets.