Feature selection in feature network models: finding predictive subsets of features with the Positive Lasso

Area of Science:

• Network analysis
• Statistical modeling
• Machine learning

Background:

• Feature network models (FNMs) represent proximity data using binary features.
• Features are crucial for constructing experimental conditions but are not always known a priori.
• Selecting an optimal feature subset is essential for balancing model fit and complexity.

Purpose of the Study:

• To develop a method for generating and selecting features when they are not known beforehand.
• To introduce a novel approach for feature selection in FNMs using Positive Lasso and Gray codes.
• To evaluate the performance of the proposed strategy for feature selection.

Main Methods:

• Utilized Gray codes for efficient feature generation, naturally linked to FNMs.
• Employed a new version of least angle regression, Positive Lasso, to restrict coefficients to be non-negative.
• Treated FNMs as univariate multiple regression models for strategy development.

Main Results:

• The proposed feature generation and selection strategy demonstrated satisfactory results for datasets with up to 22 objects.
• For datasets larger than 22 objects, the method occasionally selected more features than were truly present.
• Gray codes provide an efficient mechanism for feature generation within the FNM framework.

Conclusions:

• The developed strategy offers an effective way to generate and select features for FNMs when features are not initially known.
• The Positive Lasso method provides a robust approach to feature selection, balancing model fit and complexity.
• The method's efficacy is confirmed for smaller datasets, with limitations noted for larger, more complex scenarios.