Explanations as a New Metric for Feature Selection: A Systematic Approach.

Area of Science:

• Biomedical informatics
• Machine Learning
• Explainable Artificial Intelligence

Background:

• Machine Learning (ML) is increasingly used in biomedicine, necessitating Explainable Artificial Intelligence (XAI) for transparency and regulatory compliance.
• Feature Selection (FS) is vital in ML pipelines to reduce variables while retaining information, but its impact on model explanations is understudied.

Purpose of the Study:

• To investigate the relationship between Feature Selection (FS) methods and the explanations generated by Machine Learning (ML) models in biomedical applications.
• To develop and validate a framework for selecting optimal FS/ML models using explanation-based metrics alongside traditional performance measures.

Main Methods:

• A systematic workflow was applied to 145 datasets.
• Two explanation-based metrics (ranking and influence changes) were developed and evaluated.
• The performance of FS/ML models was assessed using accuracy, retention rate, and explanation differences with and without FS.

Main Results:

• Explanation-based metrics are complementary to accuracy and retention rate for selecting appropriate FS/ML models.
• Measuring explanation differences before and after FS is a promising approach for recommending FS methods.
• While reliefF performed well on average, optimal FS method choice is dataset-dependent.

Conclusions:

• A novel framework integrating explanation-based metrics, accuracy, and retention rate allows for a tridimensional assessment of FS methods.
• This framework aids healthcare professionals in selecting FS techniques that prioritize explainable impact, even with minor accuracy trade-offs.
• The approach enhances the selection of variables with significant explainable influence in biomedical ML applications.