An Application of Reliability Estimation in Longitudinal Designs Through Modeling Item-Specific Error Variance

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Internal consistency reliability is crucial for psychometric assessments.
• Longitudinal data presents unique challenges for reliability estimation.
• Item-specific variance can attenuate reliability coefficients.

Purpose of the Study:

• To apply Raykov's methodology for modeling item-specific variance.
• To measure internal consistency reliability using longitudinal data.
• To evaluate the impact of item specificity on reliability.

Main Methods:

• Utilized confirmatory factor analysis on data from 500 individuals across four test occasions.
• Applied composite index omega for reliability estimation.
• Corrected item error variance for item specificity.

Main Results:

• Initially low internal consistency reliability improved to acceptable levels after accounting for item-specific variance.
• Findings were robustly verified using 10,000 replicated samples.
• Percentage improvement in reliability estimates and confidence intervals were calculated.

Conclusions:

• Accounting for item-specific variance significantly enhances the accuracy of internal consistency reliability.
• Raykov's methodology provides a valuable tool for improving psychometric assessments with longitudinal data.
• This approach leads to more acceptable reliability estimates in professional qualifications testing.