Still an Ineffective Method With Supertrials/ERPs-Comments on "Decoding Brain Representations by Multimodal Learning...

Area of Science:

• Neuroscience
• Machine Learning
• Computer Vision

Background:

• A recent study proposed a novel electroencephalography (EEG) classification method for ImageNet stimuli.
• The study claimed superior performance compared to existing methods.

Purpose of the Study:

• To re-evaluate the performance of the proposed EEG classification method.
• To address potential data confounds in the original analysis.

Main Methods:

• Replication of the original analysis using a large, newly acquired dataset.
• Analysis of electroencephalography (EEG) data from subjects viewing ImageNet stimuli.
• Training and testing classifiers on aggregated supertrials.

Main Results:

• The two prior EEG classification methods achieved statistically significant above-chance accuracy.
• The newly proposed method did not achieve significant above-chance accuracy on the independent dataset.
• The original study's analysis was based on confounded data.

Conclusions:

• The proposed EEG classification method's claimed superiority is not supported by the re-analysis.
• Prior methods demonstrate robust performance in classifying EEG data for visual stimuli.
• Careful data handling and validation are crucial in machine learning for neuroscience research.