Robust Random Forest-Based All-Relevant Feature Ranks for Trustworthy AI

Area of Science:

• Machine Learning
• Bioinformatics
• Computational Statistics

Background:

• Feature selection is crucial for machine learning, especially in bioinformatics for biomarker discovery.
• Tree-based methods are commonly employed for feature selection.
• Stochastic algorithms in feature selection can lead to unstable rankings.

Purpose of the Study:

• To investigate the feature ranking instability of BORUTA, VITA, and regularized random forest (RRF) methods.
• To propose a novel approach for stabilizing feature ranks using consensus values and rank aggregation.
• To enhance the accuracy and robustness of feature selection for practical machine learning.

Main Methods:

• Studied the stability of BORUTA, VITA, and regularized random forest (RRF) feature selection algorithms.
• Investigated feature ranking instability in stochastic algorithms.
• Applied rank aggregation techniques to compute consensus values from multiple feature selection runs.

Main Results:

• Identified feature ranking instability in stochastic tree-based feature selection methods.
• Demonstrated that consensus values from multiple runs stabilize feature ranks.
• Consolidated features showed improved accuracy and robustness compared to single-run methods.

Conclusions:

• Consensus-based feature selection significantly enhances stability and reliability.
• The proposed method improves the trustworthiness of machine learning applications, particularly in biomarker discovery.
• Rank aggregation techniques offer a robust solution for feature selection instability.