Published estimates of group differences in multisensory integration are inflated

Area of Science:

• Cognitive psychology
• Neuroscience
• Psycholinguistics

Background:

• The McGurk effect, a multisensory integration illusion, is frequently used to study differences in sensory processing across various populations.
• Published research indicates variations in McGurk effect susceptibility based on age, gender, culture, language, and neurological or psychiatric conditions.

Purpose of the Study:

• To investigate the impact of publication bias on reported group differences in McGurk effect susceptibility.
• To demonstrate how selective reporting inflates effect size estimates and hinders replication.

Main Methods:

• Analysis of empirical data on the McGurk effect.
• Statistical simulations under diverse conditions to model the effects of publication bias.
• Comparison of reported effect sizes with simulated, unbiased estimates.

Main Results:

• Published estimates of group differences in the McGurk effect are significantly inflated due to the common practice of only publishing statistically significant results (p < 0.05).
• A 10% group difference, typical in many studies, can be reported as 31% when using standard sample sizes and selective reporting.
• This inflation contributes to frequent failures in replicating findings of large between-group differences.

Conclusions:

• Publication bias inflates effect sizes for the McGurk effect, leading to unreliable conclusions about sensory processing differences.
• Replication failures in studies, particularly those involving clinical populations and interventions, are exacerbated by these inflated estimates.
• To improve accuracy and replicability, research on the McGurk effect necessitates a tenfold increase in sample size compared to current practices.