Designing Optimal, Data-Driven Policies from Multisite Randomized Trials

Area of Science:

• * Educational Psychology
• * Data Science
• * Biostatistics

Background:

• * Optimal treatment regimes (OTRs) are data-driven recommendations used in computer science and personalized medicine.
• * Existing OTR research often overlooks hierarchical dependencies common in educational settings (students nested within schools).

Purpose of the Study:

• * To propose a framework for designing OTRs tailored for multisite randomized trials (MRTs) in education.
• * To adapt and evaluate Q-learning and weighting methods for improved performance in hierarchical educational data.

Main Methods:

• * Developed 12 modifications (6 Q-learning, 6 weighting) using multilevel models, moderators, and augmentations.
• * Investigated the impact of incorporating random treatment effects and cluster-level moderators.
• * Applied cluster dummies and augmentation terms within weighting methods.

Main Results:

• * All modified Q-learning methods enhanced performance in MRTs.
• * Q-learning modifications with random treatment effects excelled at handling cluster-level moderators.
• * The best-performing weighting method included cluster dummies and augmentation terms.

Conclusions:

• * The proposed framework effectively adapts OTR methods for hierarchical educational data in MRTs.
• * Modified Q-learning and weighting approaches offer improved strategies for personalized educational interventions.
• * Demonstrated application in optimizing conditional cash transfer programs to enhance educational attainment.