Related Experiment Videos
Detecting change: A comparison of three neuropsychological methods, using normal and clinical samples
R K Heaton1, N Temkin, S Dikmen
1Department of Psychiatry, University of California at San Diego, San Diego, CA 92103, USA.
Summary
Predicting neuropsychological test changes is challenging. This study found that while test-retest reliabilities are consistent, prediction models have limited generalizability, but can detect neurological changes with corrected cut-offs.
Area of Science:
- Neuropsychology
- Psychometrics
Background:
- Detecting individual patient change is crucial in neuropsychological testing.
- Limited data exists on test-retest changes and validated prediction methods are scarce.
Purpose of the Study:
- To review findings on test-retest reliabilities and practice effects.
- To examine the generalizability of prediction models to new groups.
- To explore the sensitivity of prediction models to real neurological change.
Main Methods:
- Utilized a large nonclinical sample (N=384) for test-retest reliability and practice effect analysis.
- Developed and validated prediction models for follow-up test scores.
- Assessed model sensitivity using normative cut-offs and compared complex regression with the Reliable Change Index.
Main Results:
- Reliability coefficients and practice effects showed sample similarities, but prediction methods had limited generalizability.
- Multiple significant changes were common across all groups when multiple measures were considered.
- Corrected normative cut-offs yielded modest to good sensitivity for detecting neurological recovery and deterioration.
Conclusions:
- Prediction models for neuropsychological testing show limited generalizability despite consistent reliability.
- The Reliable Change Index, adjusted for practice effects and baseline performance, is effective for detecting clinical change.
- Careful application of normative cut-offs is necessary to accurately interpret changes in neuropsychological test scores.