Beyond trial types.

Area of Science:

• Psychological research methodology
• Psychometrics
• Experimental design

Background:

• Traditional psychological experiments assume constant conditions, but controlling all variables is challenging.
• Confounding factors can violate experimental assumptions, impacting results.
• Current methods like "trial types" attempt to mitigate but are limited in scope.

Purpose of the Study:

• To propose a more general method for handling confounding variables in psychological experiments.
• To integrate the measurement of confounding factors with manipulated variables at the single-trial level.
• To improve the accuracy of effect size estimation in experimental psychology.

Main Methods:

• Utilized mathematical models within the generalized linear item response theory (GLIRT) framework.
• Treated measures of confounding factors and manipulated variables as equals on the single-trial level.
• Employed a nested likelihood ratio test for analysis, exemplified with a single-letter recognition experiment.

Main Results:

• The proposed method successfully detected a confounding interaction with time-on-task.
• This approach yielded a substantially better estimate of the main manipulation's effect size.
• Demonstrated superiority over traditional trial-type analysis in controlling for unintended variations.

Conclusions:

• The single-trial level analysis of confounding and manipulated variables offers a more robust experimental approach.
• Generalized linear item response theory provides a flexible framework for implementing this method.
• This technique enhances the reliability and validity of findings in psychological research.