Reliability Theory for Measurements with Variable Test Length, Illustrated with ERN and Pe Collected in the Flanker...

Area of Science:

• Psychophysiology
• Neuroscience
• Psychometrics

Background:

• Estimating the reliability of event-related potentials (ERPs) is essential in psychophysiology.
• The Eriksen Flanker Task and similar tests often yield unequal numbers of observations per subject due to response-locked data.
• Existing methods like generalizability theory and Bayesian estimation are complex for this specific problem.

Purpose of the Study:

• To introduce a simpler, classical test theory-based approach for estimating ERP reliability.
• To address the challenge of unequal observations per subject in psychophysiological data.
• To provide a single, overall reliability coefficient for groups with varying observation counts.

Main Methods:

• Application of classical test theory and frequentist estimation techniques.
• Development of a novel method for calculating reliability with unbalanced data.
• Comparison with generalizability theory and Bayesian estimation approaches.

Main Results:

• The proposed classical approach effectively estimates ERP reliability, comparable to existing methods.
• This method offers a simpler framework and additional insights not found in generalizability theory.
• A new definition for an overall reliability coefficient for subjects with unequal observations was established.

Conclusions:

• Classical test theory provides a viable and simpler alternative for estimating ERP reliability with unequal data.
• The new approach offers practical advantages and deeper understanding compared to generalizability theory.
• While favoring the classical approach, the study acknowledges the utility of generalizability theory.