Related Experiment Video
Updated: Apr 17, 2026
08:35
An Operant Intra-/Extra-dimensional Set-shift Task for Mice
Published on: January 22, 2016
12.9K
Overall performance of Oort's procedure for response shift detection at item level: a pilot simulation study
Antoine Vanier1, Véronique Sébille, Myriam Blanchin
1EA 4275 Biostatistics Pharmacoepidemiology and Subjective Measures in Health Sciences, LUNAM, University of Nantes, Nantes, France, antoine.vanier@psl.aphp.fr.
Summary
This study evaluated Oort's procedure (OP) for detecting response shift (RS). The likelihood-ratio test (LRT) demonstrated superior power for RS detection compared to information criteria (IC).
Area of Science:
- Psychometrics
- Statistical Modeling
- Survey Methodology
Background:
- Response shift (RS) poses a challenge in longitudinal studies, affecting the interpretation of change over time.
- Oort's procedure (OP) is a statistical method for detecting RS, but its performance characteristics require thorough investigation.
- The choice of statistical criterion for assessing global RS occurrence can impact study findings.
Purpose of the Study:
- To evaluate the performance of Oort's procedure (OP) for response shift (RS) detection using simulation.
- To compare the effectiveness of the likelihood-ratio test (LRT) against various information criteria (IC) for global RS assessment.
- To examine the influence of sample characteristics on the type I error rate and statistical power of RS detection methods.
Main Methods:
- Simulated responses to five binary items across two measurement occasions.
- Investigated 36 combinations of sample characteristics, including sample size, true change, variable correlations, and uniform recalibration RS.
- Generated 1000 datasets per condition, applying OP for RS detection and assessing type I error, power, and overall performance.
Main Results:
- Information criteria (IC) yielded type I error rates below 5%, while the likelihood-ratio test (LRT) was close to 5%.
- The LRT exhibited higher statistical power for RS detection compared to all tested ICs, including the Akaike Information Criterion (AIC).
- LRT power exceeded 90% across most simulated conditions, with notable exceptions at smaller sample sizes (n=100) or specific RS configurations.
Conclusions:
- The LRT demonstrates superior performance in terms of statistical power for global RS assessment compared to information criteria.
- The findings support Oort's original recommendation to utilize the LRT as the primary criterion for evaluating global response shift.
- Careful consideration of sample characteristics is essential when interpreting the results of RS detection procedures.
