Uncovering individualised treatment effects for educational trials

Area of Science:

• Educational research
• Health intervention studies
• Applied machine learning

Background:

• Randomised Controlled Trials (RCTs) are standard for evaluating school interventions, typically reporting Average Treatment Effects (ATE).
• Key decisions often require understanding effects on individuals, not just averages.
• Current subgroup analyses lack standardization and can yield misleading results.

Purpose of the Study:

• To develop and deploy an alternative to ATE and subgroup analysis for evaluating interventions.
• To predict Individualised Treatment Effects (ITEs) using a machine-learning and regression-based framework.
• To provide a more nuanced understanding of intervention effectiveness for specific individuals.

Main Methods:

• Developed a novel machine-learning and regression-based framework.
• Applied the framework to analyze data from 48 Education Endowment Foundation (EEF)-funded RCTs.
• Evaluated the framework's ability to predict Individualised Treatment Effects (ITEs).

Main Results:

• The developed framework offers an alternative to traditional Average Treatment Effects (ATE) and subgroup analyses.
• Individualised Treatment Effects (ITEs) can be predicted, identifying specific individuals who benefit from interventions.
• Demonstrated the framework's utility across a large dataset of over 200,000 students.

Conclusions:

• Individualised Treatment Effects (ITEs) provide more granular insights than ATE or subgroup analyses.
• The machine-learning framework facilitates targeted decision-making in education, healthcare, and other child-focused fields.
• This approach enhances the precision and applicability of intervention research findings.