One model may not fit all: Subgroup detection using model-based recursive partitioning

Area of Science:

• Statistics
• Educational Measurement
• Psychometrics

Background:

• Model-based recursive partitioning (MOB) offers a flexible framework for analyzing heterogeneity in parametric models.
• Detecting and explaining subgroup differences is crucial in fields like intervention and educational studies.
• Existing methods may not fully capture complex heterogeneity patterns in mixed-effects or item response theory models.

Purpose of the Study:

• To introduce the general Model-based recursive partitioning (MOB) framework.
• To illustrate the application of MOB-based methods for detecting and explaining heterogeneity in generalized linear mixed models (GLMM) and item response theory (IRT) frameworks within educational research.
• To demonstrate the utility of GLMM trees and Rasch trees for subgroup detection.

Main Methods:

• The study utilizes Model-based recursive partitioning (MOB) as a general framework.
• Specific MOB extensions, GLMM trees and Rasch trees, are applied to analyze heterogeneity.
• A novel stopping criterion for Rasch trees, based on differential item functioning (DIF) effect sizes, is employed.

Main Results:

• GLMM trees successfully identified subgroups with differing parameters in mixed-effects models, applied to longitudinal Head Start data to reveal performance gain variations.
• Rasch trees detected subgroups exhibiting differential item functioning (DIF) in IRT models, highlighting the importance of DIF analysis before group comparisons.
• The new stopping criterion effectively guided subgroup detection based on DIF effect sizes.

Conclusions:

• MOB provides a powerful and versatile tool for uncovering and interpreting subgroup heterogeneity in educational studies.
• GLMM trees and Rasch trees are effective extensions of MOB for analyzing complex data structures in mixed-effects models and IRT.
• The integration of a DIF-based stopping criterion enhances the practical application of Rasch trees for robust subgroup analysis.