Probing machine-learning classifiers using noise, bubbles, and reverse correlation.

Area of Science:

• Neuroscience
• Machine Learning
• Computational Neuroscience

Background:

• Machine learning (ML) tools are increasingly used in scientific fields for complex classification tasks.
• In neuroscience, ML classifiers aid in medical image diagnosis, signal monitoring, and neural decoding.
• A significant limitation of current ML tools is their 'black-box' nature, lacking interpretability, which has ethical and scientific implications.

Purpose of the Study:

• To propose a simple, versatile method for characterizing the information used by ML classifiers.
• To enhance the interpretability of ML models in neuroscience and other scientific domains.
• To bridge the gap between neuroscientists and complex machine-learning tools.

Main Methods:

• The proposed method involves feeding noisy versions of training or custom-generated samples into a classifier.
• Both multiplicative ('bubbles') and additive noise are applied to the input data.
• Reverse correlation techniques are adapted to extract discriminative and represented information from the classifier's decisions.

Main Results:

• The method was successfully demonstrated on diverse classification tasks: number recognition (CNN), speech/music classification (SVM), and sleep stage classification (Random Forest).
• In all tested cases, the features extracted by the method were readily interpretable.
• Quantitative comparisons show the method's interpretation performance matches state-of-the-art techniques for Convolutional Neural Networks (CNNs).

Conclusions:

• The developed method provides an intuitive and versatile approach to interpreting ML classifier behavior.
• It leverages the established reverse correlation framework, making it familiar and accessible to neuroscientists.
• The method's generic nature allows application to any classifier and input data type, offering broad utility.