Beyond matching on the mean in developmental disabilities research

Area of Science:

• Developmental Psychology
• Biostatistics
• Educational Measurement

Background:

• Group matching is standard for controlling confounding variables in developmental disabilities research.
• Equivalency of means is routinely checked, but variance homogeneity and distribution shapes are often overlooked.
• Inadequate matching can lead to inaccurate research findings.

Purpose of the Study:

• To highlight the critical need to assess distributional properties beyond means in group matching.
• To address the issue of item response profile equivalency for robust matching.
• To introduce statistical and graphical methods for improved group matching in research.

Main Methods:

• Utilizing statistical indices and graphical displays to assess distribution shapes and variances.
• Examining item response profiles for concordance between overall and item-level scores.
• Describing Angoff's Delta plot method for detecting differential item functioning.

Main Results:

• Group matching can be significantly flawed if distributional concerns are not addressed.
• Mismatched item response profiles (molar vs. molecular levels) indicate unsatisfactory matching.
• Angoff's Delta plot method offers a simple approach for assessing test/item correspondence.

Conclusions:

• Robust group matching necessitates evaluating variance homogeneity and distribution shapes.
• Concordance between overall and item-level scores is crucial for valid matching.
• Angoff's Delta plot method is a valuable tool for post hoc item-level analysis and ensuring matching quality.