Estimating sparse regression models in multi-task learning and transfer learning through adaptive penalisation.

Area of Science:

• Machine Learning
• Statistics

Background:

• High-dimensional data presents challenges in prediction and classification tasks.
• Sharing information between related problems can improve model performance and interpretability.

Purpose of the Study:

• To propose a novel two-stage procedure for information sharing between related high-dimensional prediction or classification problems.
• To develop a method that enhances feature selection, effect direction, and effect size estimation.

Main Methods:

• A two-stage sparse regression procedure is employed, performing regression separately for each problem.
• The first stage uses no prior information, while the second stage utilizes coefficients from the first stage as prior information.
• Feature-specific and sign-specific adaptive weights are designed to facilitate information sharing.

Main Results:

• The proposed approach is applicable to multi-task learning and transfer learning.
• It yields sparse and interpretable models with few non-zero coefficients.
• Simulations and applications demonstrate reduced feature selection while maintaining comparable predictive performance.

Conclusions:

• The developed method offers an effective way to share information across related high-dimensional problems.
• It provides a practical solution for improving model efficiency and interpretability in machine learning.
• An R package 'sparselink' is available for implementation.