Within-participant statistics for cognitive science

Area of Science:

• Cognitive Science
• Experimental Psychology
• Behavioral Research

Background:

• Traditional cognitive science research emphasizes population average effects.
• This focus may obscure individual variability and the generalizability of findings.
• An alternative approach is needed to better understand individual responses.

Purpose of the Study:

• To introduce and advocate for the 'participant replication probability' as a key metric in cognitive science.
• To highlight the conceptual and practical advantages of focusing on individual effects over population averages.
• To propose a shift in experimental design and analysis towards individual-level quantification.

Main Methods:

• Shifting focus from population means to individual participant data.
• Quantifying the proportion of individuals exhibiting a specific cognitive effect (prevalence).
• Analyzing experimental results at the individual level to determine participant replication probability.

Main Results:

• The participant replication probability offers a more direct measure of effect prevalence.
• This metric provides a clearer understanding of how consistently an effect manifests across individuals.
• The approach enhances the interpretability and practical relevance of experimental findings.

Conclusions:

• The participant replication probability represents a valuable alternative to traditional population average analysis.
• Adopting this individual-focused approach can lead to more robust and generalizable cognitive science findings.
• This method offers significant conceptual and practical advantages for designing and interpreting experiments.