An efficient model selection for linear discriminant function-based recursive feature elimination.

Area of Science:

• Machine Learning
• Bioinformatics
• Computational Biology

Background:

• Model selection is crucial for Support Vector Machine-based Recursive Feature Elimination (SVM-RFE).
• Estimating generalization error for linear SVM-RFE is challenging, hindering effective model selection.
• Tuning the penalty parameter C is critical for SVM-RFE performance.

Purpose of the Study:

• To propose an approximation method for evaluating the generalization error of linear SVM-RFE.
• To design a novel criterion for tuning the penalty parameter C in linear SVM-RFE.
• To reduce the computational cost of the proposed algorithm through alpha seeding approaches.

Main Methods:

• Developed an approximation method to estimate generalization error in linear SVM-RFE.
• Introduced a new criterion for optimizing the penalty parameter C.
• Implemented alpha seeding strategies to enhance computational efficiency.

Main Results:

• The proposed algorithm demonstrated superior performance compared to existing methods on bioinformatics datasets.
• Alpha seeding approaches significantly reduced the computational time required by the algorithm.
• Empirical evidence supports the effectiveness of the new generalization error estimation and parameter tuning method.

Conclusions:

• The novel approach enhances model selection for linear SVM-RFE by providing a reliable generalization error estimate.
• The proposed criterion effectively tunes the penalty parameter C, leading to improved model performance.
• Alpha seeding successfully mitigates the computational expense, making the method more practical for large-scale applications.