Using small-scale randomized controlled trials to evaluate the efficacy of new curricular materials

Area of Science:

• Educational research
• Curriculum development
• Science education

Background:

• Conducting rigorous evaluations in K-12 settings presents challenges due to classroom constraints and limited funding.
• There is a need for practical models of scientifically sound research applicable to school environments.
• University science faculty often seek ways to implement rigorous evaluations within K-12 outreach.

Purpose of the Study:

• To evaluate the efficacy of supplemental curricular materials on epigenetics understanding in high school students.
• To explore the feasibility of implementing a small-scale randomized controlled trial (RCT) in a K-12 public school.
• To provide a case study for faculty on conducting rigorous educational evaluations with limited scope and budget.

Main Methods:

• A small-scale randomized controlled trial (RCT) was conducted with 145 diverse public high school students.
• Students were randomly assigned to either a treatment group (epigenetics supplemental materials) or a comparison group (alternative activities).
• Student understanding of epigenetics was assessed using a posttest to compare the two conditions.

Main Results:

• Students in the treatment condition demonstrated significantly higher posttest scores compared to the comparison group.
• An effect size of Cohen's d = 0.40 indicated a moderate positive impact of the supplemental materials.
• The study successfully implemented a rigorous evaluation design within a real-world K-12 classroom setting.

Conclusions:

• Supplemental curricular materials focused on epigenetics can effectively enhance high school students' understanding.
• Small-scale RCTs are feasible and valuable for evaluating educational interventions in K-12 contexts.
• The study offers practical insights and recommendations for researchers conducting similar evaluations in educational settings.