Can One Pool Over Site in a Multi-Site Study With Categorical Item Measuring Instruments? A Multiple Testing...

Area of Science:

• Educational Research
• Behavioral Science
• Psychometrics

Background:

• Multi-site studies are common in educational and behavioral research.
• Combining data across sites requires careful validation.
• Existing methods may not adequately address site-specific response variations.

Purpose of the Study:

• To present a statistical procedure for assessing data collapsibility across multiple research sites.
• To enable the pooling of data from diverse study locations.
• To provide a method applicable to polytomous items in multi-component instruments.

Main Methods:

• The core method involves testing for cross-site identity of response distributions.
• It utilizes statistical tests suitable for polytomous item data.
• The procedure is designed for practical application in empirical research.

Main Results:

• The proposed procedure effectively determines if data can be pooled across sites.
• It confirms the generalizability of the method to various item types.
• The technique is demonstrated using data from a child development survey.

Conclusions:

• The outlined procedure offers a robust way to examine data collapsibility in multi-site research.
• It facilitates more efficient and powerful data analysis by allowing data pooling.
• The method is accessible through widely available statistical software.