Intervention in prediction measure: a new approach to assessing variable importance for random forests

Area of Science:

• Machine Learning
• Statistical Modeling
• Bioinformatics

Background:

• Random forests are widely used for complex data analysis due to their flexibility.
• Existing variable importance measures often depend on performance metrics, which can be ambiguous.
• Multivariate response random forests present unique challenges for importance assessment.

Purpose of the Study:

• Introduce and evaluate a novel variable importance measure, the Intervention in Prediction Measure.
• Compare the new measure against established methods in various data scenarios.
• Assess the applicability and benefits of the new measure in bioinformatics.

Main Methods:

• Developed the Intervention in Prediction Measure, independent of performance metrics.
• Conducted simulation studies for classification with mixed-type and correlated predictors.
• Applied the measure to multivariate response problems and real-world bioinformatics datasets.

Main Results:

• The Intervention in Prediction Measure demonstrated strong competitiveness in simulations.
• The new measure improved performance in two established bioinformatics applications.
• Effectiveness shown across diverse contexts including mixed-type and correlated predictors.

Conclusions:

• The Intervention in Prediction Measure is highly interpretable, expressed as a percentage.
• It is versatile, applicable to global, class-specific, and case-wise analyses.
• The measure supports any response type, algorithm, and can be used alongside existing methods.