Computer-assisted learning in the real world: How Khan Academy influences student math learning

Area of Science:

• Educational Technology
• Learning Sciences
• Applied Mathematics

Background:

• Computer-assisted learning (CAL) is a cost-effective method for implementing mastery learning.
• Previous research on CAL's effectiveness often uses experimental conditions difficult to replicate in real classrooms.

Purpose of the Study:

• To evaluate the real-world impact of CAL on student math performance.
• To provide policy-relevant estimates of CAL's effectiveness at various usage levels.

Main Methods:

• Utilized a three-year panel of administrative data from over 200,000 students.
• Employed within-teacher and within-school variations in classroom CAL practice time to identify causal effects.

Main Results:

• A gain of 0.031 standard deviation (SD) in math test scores was observed with 6.6 hours of annual Khan Academy practice.
• Projected gains reached 0.085 SD at the recommended 30 minutes per week, showing approximately linear increases.
• Higher-achieving students benefited more, spending more time on CAL and mastering more skills.

Conclusions:

• CAL demonstrates measurable positive effects on math performance in typical classroom settings.
• Encouraging productive CAL use, focused on skill mastery, can help reduce achievement gaps and enhance overall student gains, particularly for struggling learners.