Safe RuleFit: Learning Optimal Sparse Rule Model by Meta Safe Screening.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Learning sparse rule models involves selecting optimal rules from a vast search space.
• Existing methods face computational intractability due to the large number of potential rules.

Purpose of the Study:

• To propose Safe RuleFit (SRF), a novel framework for learning optimal sparse rule models.
• To introduce meta safe screening (mSS) for efficient multi-feature screening in rule model optimization.

Main Methods:

• Developed meta safe screening (mSS), an extension of safe screening (SS) techniques.
• SRF framework leverages mSS to exploit hyper-rectangle inclusion relations for feature screening.
• Applied SRF to regression and classification tasks, with potential for group regularization.

Main Results:

• SRF effectively addresses the computational challenges in learning sparse rule models.
• mSS enables efficient screening of multiple features, improving scalability.
• Intensive numerical experiments demonstrate the advantages of the SRF framework.

Conclusions:

• SRF provides a computationally tractable and general framework for sparse rule model learning.
• The proposed mSS technique significantly enhances the efficiency of feature selection.
• SRF offers a promising approach for both regression and classification problems.