Principles of defining reliable change indices

Area of Science:

• Psychometrics
• Statistical Methods

Background:

• Determining reliable change is crucial in various fields, including psychology and education.
• Existing methods for assessing reliable change have been debated, with newer indices often favored over classic approaches.

Purpose of the Study:

• To scrutinize salient measures for determining reliable change.
• To compare the classic null hypothesis method with interval estimation procedures, such as Kelley's formula.
• To re-evaluate the standing of the classic null hypothesis method in light of newer indices.

Main Methods:

• Comparative analysis of statistical methods for reliable change.
• Examination of interval estimation procedures and Kelley's formula.
• Expansion of Kelley's formula to a null hypothesis framework, incorporating stochastic sample information.

Main Results:

• Interval estimation methods, including Kelley's formula, present significant drawbacks for determining reliable change.
• When Kelley's formula is correctly adapted to a null hypothesis method, it demonstrates the classic method's nature as a large sample approximation.
• The classic null hypothesis method is robust and not inferior to newer indices.

Conclusions:

• The classic null hypothesis method for reliable change is a valid and often preferred approach, especially as a large sample approximation.
• Authors proposing new indices have often undervalued the classic method.
• Reliable change assessment requires careful consideration of statistical properties and potential drawbacks of different methods.