Meta-analyzing individual participant data from studies with complex survey designs: A tutorial on using the...

Area of Science:

• Behavioral, social, economic, and health sciences research.
• Statistical methodology for analyzing complex survey data.

Background:

• Descriptive analyses using representative individual participant data (IPD) are crucial for understanding social phenomena.
• Meta-analytic integration of IPD from complex surveys offers strong empirical evidence for consistency and generalizability.
• Educational large-scale assessments (ELSAs) and social, health, and economic surveys present unique statistical challenges.

Approach:

• Presents a two-stage approach for IPD meta-analysis tailored for complex survey designs.
• Stage 1 involves descriptive analyses accounting for sampling weights, clustering, and missing data.
• Stage 2 integrates results using three-level meta-analytic and meta-regression models to handle dependencies among effect sizes.

Key Points:

• Illustrates the two-stage approach using Programme for International Student Assessment (PISA) reading achievement data.
• Demonstrates analysis of standardized mean differences (e.g., gender differences), correlations (e.g., with socioeconomic status [SES]), and interactions.
• Provides R scripts and datafiles for reproducible research, applicable to various complex survey studies.

Conclusions:

• The two-stage IPD meta-analysis approach effectively handles statistical complexities in large-scale survey data.
• This methodology enhances the reliability and generalizability of findings across diverse research fields.
• The tutorial's guidance is valuable for synthesizing evidence from educational, social, health, and economic surveys.