Using Meta-Analysis and Propensity Score Methods to Assess Treatment Effects Toward Evidence-Based Practice in...

Area of Science:

• Second Language Acquisition
• Educational Psychology
• Applied Linguistics

Background:

• Assessing treatment effects in second language (L2) teaching requires robust evidence-based practices.
• Previous research on extensive reading has faced methodological limitations, particularly in establishing appropriate control groups.
• Effect sizes in L2 teaching studies vary due to learner, measure, and teaching approach differences.

Purpose of the Study:

• To estimate the effect sizes of extensive reading on L2 reading comprehension.
• To address methodological restrictions in prior research, specifically selection bias.
• To provide reliable estimates for implementing evidence-based L2 teaching practices.

Main Methods:

• Study 1: Retrospective meta-analysis of studies with equivalent treatment and control groups.
• Study 2: Propensity score analysis (nearest neighborhood matching, full matching) to minimize selection bias in non-randomized groups.
• Data collected from Japanese university students (N=224) in extensive reading and control classes.

Main Results:

• Initial meta-analysis predicted an effect size (d=0.55) for reading comprehension, adjusted to 0.37 after accounting for selection bias.
• Propensity score analysis yielded average treatment effects ranging from d=0.24-0.44 for all participants and d=0.32-0.40 for treated individuals.
• Results from Study 2 were consistent with the benchmark established in Study 1.

Conclusions:

• Extensive reading demonstrates a moderate, positive effect on L2 reading comprehension.
• Methodological refinements, particularly propensity score analysis, are crucial for accurate effect size estimation.
• Findings support the implementation of extensive reading as an evidence-based L2 teaching practice.