The Design of Making Pre-K count and High Fives: Two-Stage, Multiyear Random Assignment at Different Levels.

Area of Science:

• Early Childhood Education
• Educational Psychology
• Quantitative Research Methods

Background:

• Evaluating aligned math programs in early childhood settings requires robust research designs.
• Previous research often faces challenges in implementing complex multi-stage study designs.
• The Making Pre-K Count and High 5s studies utilized a phased two-stage, multi-level design.

Purpose of the Study:

• To describe implementation challenges of a phased two-stage, multi-level design in early childhood math programs.
• To detail approaches used to resolve these implementation challenges.
• To present sensitivity analyses examining the robustness of study findings.

Main Methods:

• Phased two-stage, multi-level design involving random assignment at both pre-kindergarten and kindergarten levels.
• Implementation of two aligned math programs: Making Pre-K Count (curriculum + professional development) and High 5s (supplemental math clubs).
• Assessment of children's math skills using the Research-Based Early Math Assessment-Kindergarten (REMA-K) and Woodcock-Johnson Applied Problems test.

Main Results:

• The phased multi-armed design, though complex, balanced statistical power, research questions, and resource efficiency.
• Sensitivity analyses indicated that the design successfully created meaningfully and statistically equivalent groups.
• The design facilitated a flexible and expansive research study while managing logistical and analytical complexities.

Conclusions:

• The phased two-stage, multi-level design is a viable, albeit complex, approach for evaluating early childhood interventions.
• Careful consideration of logistical and analytical challenges is crucial for successful implementation.
• This design offers a robust framework for examining the effects of aligned educational programs on children's math skills.