A Note on the Structural Change Test in Highly Parameterized Psychometric Models.

Area of Science:

• Psychometrics and Statistical Modeling
• Behavioral and Social Sciences Research

Background:

• Parameter invariance across subgroups is crucial for statistical tests, impacting replicability and theory.
• Ignoring subgroup differences threatens study validity and model specification.
• Structural change tests assess parameter invariance but rely on asymptotic assumptions.

Purpose of the Study:

• To address the limitations of traditional structural change tests in small samples and large psychometric models.
• To introduce and evaluate permutation approaches as an alternative for obtaining sampling distributions.
• To enhance the power and validity of structural change tests.

Main Methods:

• Investigated the empirical fluctuation process in structural change tests.
• Analyzed the deviation of the empirical fluctuation process from Brownian bridge in small samples.
• Implemented and compared permutation approaches against standard asymptotic approximations for sampling distribution estimation.

Main Results:

• The empirical fluctuation process deviates from Brownian bridge in small samples, especially in large psychometric models.
• Standard methods for obtaining sampling distributions are invalid, leading to conservative structural change tests.
• Permutation approaches provide a valid estimation of the sampling distribution, avoiding distributional assumptions and improving test power.

Conclusions:

• Permutation approaches offer a superior alternative to asymptotic approximations for structural change tests.
• Resampling methods enhance statistical power and address limitations of traditional inference in psychometric modeling.
• The study advocates for permutation-based methods to improve the reliability and validity of structural change tests.