Drawing Generalizable Conclusions From Multilevel Models: Commentary on Van de Calseyde and Efendić (2022)

Area of Science:

• Cognitive Psychology
• Social Psychology
• Behavioral Economics

Background:

• Inner-crowd wisdom, the improvement from aggregating two estimates from one person, was previously thought to be enhanced by perspective-taking.
• Van de Calseyde and Efendić (2022) proposed that adopting a disagreeing viewpoint boosts this effect.

Purpose of the Study:

• To reanalyze the data presented by Van de Calseyde and Efendić (2022).
• To investigate the statistical validity of the claim that perspective-taking enhances inner-crowd wisdom.

Main Methods:

• Reanalysis of existing dataset using multilevel modeling.
• Examination of item-level effects and variance components.
• Comparison of random-intercept models with models accounting for experimental condition.

Main Results:

• The apparent benefit of perspective-taking in the original study was found to be an artifact.
• Anticonservative multilevel models, specifically random-intercept models, led to an underestimation of item-level variance.
• Failure to account for item-level effects of experimental condition created an illusory finding.

Conclusions:

• The original conclusion that adopting a disagreeing perspective enhances inner-crowd wisdom is not supported by the reanalysis.
• Methodological limitations in the original statistical analysis likely produced spurious results.
• Further research should employ more rigorous statistical methods to investigate the effects of perspective-taking on judgment aggregation.