Experiments with More Than One Random Factor: Designs, Analytic Models, and Statistical Power.

Area of Science:

• Psychological research methodology
• Statistical modeling in behavioral science

Background:

• Traditional analysis of psychological experiments often assumes a single random factor (e.g., participants).
• Many studies incorporate multiple random factors (participants and stimuli), where traditional methods can introduce bias.
• Existing analytical approaches may not adequately address complex experimental designs with multiple random factors.

Purpose of the Study:

• To present a comprehensive typology of experimental designs with two random factors (crossed or nested) and one fixed factor.
• To introduce appropriate linear mixed models and effect size measures for these complex designs.
• To provide tools for power estimation and guide design choices for enhanced replicability.

Main Methods:

• Development of a typology for designs with crossed and nested random factors.
• Application of linear mixed models tailored to designs with multiple random factors.
• Creation of effect size measures and power estimation tools for complex designs.

Main Results:

• Established a framework for analyzing psychological data with multiple random factors.
• Demonstrated the potential for bias in traditional methods when applied to such designs.
• Provided practical tools for appropriate statistical analysis and design selection.

Conclusions:

• Linear mixed models offer a robust solution for analyzing psychological experiments with multiple random factors.
• Appropriate statistical methods are crucial for reducing bias and increasing the replicability of research findings.
• This work encourages the adoption of advanced analytical techniques for more accurate psychological research.