Clarifying problems and offering solutions for correlated error when assessing the validity of selected-subtest short...

Area of Science:

• Psychometrics
• Psychological Assessment
• Statistical Methods

Background:

• Evaluating the validity of short-form (SF) tests against their full-scale (FS) counterparts is crucial in psychometric research.
• Current methods for correcting overlapping error variance in SF validity coefficients are often misapplied, leading to overattenuation.

Purpose of the Study:

• To clarify the correct application of statistical formulas for correcting SF validity coefficients.
• To propose a modified equation for calculating SF validity that accounts for error variance more accurately.
• To provide a method for obtaining both liberal and conservative validity estimates.

Main Methods:

• The study clarifies that corrections for overlapping error variance should utilize subtest-level versus full-scale (FS)-level data.
• A modified equation incorporating FS-level scores is proposed for validity coefficient estimation.
• The proposed method was demonstrated using data from a large normative sample (N = 2,450) and a clinical psychiatric sample (N = 216).

Main Results:

• Previous applications of P. Levy's (1967) formula for correcting SF validity have led to overattenuation of the validity coefficient.
• The proposed modified equation provides a more accurate estimation of SF validity by correctly accounting for error variance.
• The method yields comparable liberal and conservative validity measures, facilitating robust comparisons across estimation techniques.

Conclusions:

• The standard approach to correcting SF validity coefficients is flawed due to misapplication of established formulas.
• A revised statistical approach using full-scale scores is recommended for more accurate SF validity estimation.
• The proposed modified equation offers a practical tool for researchers and clinicians evaluating SF test validity in diverse populations.