Interaction effect: Are you doing the right thing?

Area of Science:

• Neuroscience
• Statistics
• Scientific methodology

Background:

• Interaction effects are crucial in understanding complex relationships in scientific research, particularly in neuroscience.
• Despite extensive discussion, misinterpretations of interaction effects persist across scientific disciplines, including neuroscience.
• A review of recent neuroscience literature revealed that post-hoc pairwise comparisons are overwhelmingly used to interpret significant interaction effects.

Purpose of the Study:

• To identify and explain the limitations of using post-hoc pairwise comparisons for interpreting interaction effects.
• To propose and discuss superior methods for the accurate interpretation of interaction effects.
• To provide practical guidance and reproducible resources for researchers.

Main Methods:

• Systematic review of 645 neuroscience papers published between 2019 and 2020.
• Analysis of interpretation methods for statistically significant interaction effects (N=221).
• Comparative discussion of interpretation techniques, including explorative and model selection procedures.

Main Results:

• 93.2% of studies reporting significant interaction effects utilized post-hoc pairwise comparisons for interpretation.
• This prevalent method is shown to be a common source of misinterpretation.
• Alternative, more robust interpretation strategies are presented.

Conclusions:

• The common reliance on post-hoc pairwise comparisons for interaction effects in neuroscience leads to frequent misinterpretations.
• Researchers should adopt more effective methods, such as explorative and model selection procedures, for accurate interpretation.
• Accessible examples and materials are provided to facilitate the correct application of these methods.