Stimulus-specific random effects inflate false-positive classification accuracy in...

Area of Science:

• Neuroimaging analysis
• Machine learning in neuroscience

Background:

• Multivariate-voxel pattern analysis (MVPA) is a common neuroimaging technique.
• Current by-subject analysis methods in MVPA may lead to inflated Type-1 error rates.
• This inflation is due to unaddressed variance from random stimulus effects common across subjects.

Purpose of the Study:

• To identify the limitations of traditional by-subject analysis in MVPA.
• To propose and validate an alternative statistical approach for evaluating MVPA accuracy.
• To demonstrate the impact of random stimulus effects on statistical inference in MVPA.

Main Methods:

• Statistical simulations were used to compare analysis methods.
• Real functional magnetic resonance imaging (fMRI) data were analyzed.
• Generalized linear mixed-effects modeling was implemented as a proposed solution.
• By-subject analysis was performed as the traditional comparison method.

Main Results:

• The traditional by-subject analysis demonstrated significantly increased Type-1 error rates.
• Generalized linear mixed-effects modeling successfully maintained nominal Type-1 error rates.
• The proposed method accounts for random stimulus effects, improving statistical validity.

Conclusions:

• By-subject analysis in MVPA inflates Type-1 error rates and should be used with caution.
• Generalized linear mixed-effects modeling provides a statistically sound alternative for MVPA accuracy evaluation.
• Incorporating random stimulus effects is crucial for accurate statistical inference in MVPA.