Hypothesis Testing Methods for Multivariate Multi-Occasion Intra-Individual Change.

Area of Science:

• Psychological Measurement
• Educational Measurement
• Statistical Modeling

Background:

• Assessing individual change over time is crucial in psychological and educational fields.
• Existing methods may not adequately capture changes across multiple latent traits and occasions.

Purpose of the Study:

• To introduce and evaluate novel statistical methods for assessing individual change on multiple latent traits across multiple measurement occasions.
• To compare the performance of these methods using simulation studies and real-world data.

Main Methods:

• Introduced four statistical methods: likelihood ratio test (LRT), multivariate Wald test (MWT), modified multivariate Wald test (MMWT), and score test (ST).
• Conducted simulation studies manipulating number of occasions, change magnitudes, change patterns, and latent trait correlations.
• Evaluated true positive rate (TPR) and false positive rate (FPR) under fixed-form and computerized adaptive testing (CAT) conditions.

Main Results:

• All methods except MWT showed good control of the false positive rate (FPR).
• The likelihood ratio test (LRT) is recommended for balancing FPR and TPR.
• Larger change magnitudes increased the true positive rate (TPR).
• Computerized adaptive testing (CAT) yielded higher TPR than conventional tests for the same test length.

Conclusions:

• The proposed methods, particularly LRT, effectively identify psychometrically significant individual change across multiple latent traits and occasions.
• Real-data application demonstrated agreement among methods and identified patients with substantial profile differences.
• The study supports the valid performance of the novel methods for detecting individual change.