Reliability Theory for Measurements with Variable Test Length, Illustrated with ERN and Pe Collected in the Flanker...

Area of Science:

• Psychophysiology
• Neuroscience
• Psychometrics

Background:

• Estimating the reliability of event-related potentials (ERPs) is crucial in psychophysiology.
• The Eriksen Flanker Task often yields unequal numbers of observations per subject due to response-dependent neurological reactions.
• Existing methods like generalizability theory and Bayesian estimation present complexities.

Purpose of the Study:

• To introduce and evaluate a novel approach for estimating ERP reliability using classical test theory and frequentist estimation.
• To address the challenge of unequal observations per subject in reliability analyses.
• To offer a simpler and potentially more insightful alternative to existing methods.

Main Methods:

• Application of classical test theory principles to ERP data.
• Utilizing frequentist estimation techniques for reliability assessment.
• Development of a new overall reliability coefficient for groups with unequal observations.

Main Results:

• The proposed classical test theory approach effectively estimates ERP reliability.
• This method provides a simpler framework compared to generalizability theory.
• The approach offers additional insights into reliability aspects previously unresolved.
• A novel reliability coefficient for unequally observed subjects was defined.

Conclusions:

• Classical test theory offers a viable and simpler alternative for estimating ERP reliability, especially with unequal observations.
• The new frequentist approach provides valuable insights and a unified reliability coefficient.
• While generalizability theory has its merits, the classical approach presents a compelling alternative for specific psychophysiological research questions.