Meta-analysis of response time data in a single experiment

Area of Science:

• Cognitive Psychology
• Psychometrics
• Behavioral Neuroscience

Background:

• Conventional analysis of reaction time data often uses mean response times, which can be imprecise.
• This approach overlooks variations in data precision across individual means, leading to equal weighting.
• A more robust analytical method is needed to fully leverage reaction time data.

Purpose of the Study:

• To apply a within-study meta-analysis to reaction time data from the Stroop color-naming task.
• To compare the efficacy of this meta-analysis approach against conventional mean-based analyses.
• To assess the influence of between-subject factors on effect sizes and significance levels.

Main Methods:

• A meta-analysis was performed on individual trial reaction time data from a single experiment with a mixed design.
• Each subject was treated as an independent replication.
• Time-series regression with lag 1 autocorrelation adjustment was used for individual subject data, followed by combining results for overall effect size and significance.

Main Results:

• The meta-analysis of individual trial reaction times provided a more powerful test of research hypotheses compared to conventional mean reaction time analysis.
• The within-study meta-analysis effectively accounted for differences in precision among individual means.
• Analysis of a between-subject factor revealed its impact on effect sizes and significance.

Conclusions:

• Within-study meta-analysis of individual trial reaction times is a more powerful statistical approach than traditional mean-based methods.
• This technique offers advantages in analyzing Stroop color-naming task data and potentially other reaction time studies.
• Further exploration of the advantages and limitations of this meta-analytic strategy is warranted.