Statistical methods and reporting practices in the study of dyslexia

This report presents findings from a review of the journals Dyslexia and Annals of Dyslexia over a 10-year period from 2005 to 2014. Three hundred articles were reviewed to examine statistical methodology and reporting practices. Articles were coded according to their reported covariates, effect sizes, and statistical methods. In addition, trends were examined in relation to the 2008 release of the What Works Clearinghouse Procedures and Standards Handbook from the Institute of Education Sciences in order to evaluate any potential impact of recommended practices. We found an increase in prevalence of specific statistical methods as well as increases in reporting of effect sizes following publication of the handbook. The reporting of effect sizes statistically significantly increased following publication of the manual (55% prepublication vs. 72% post-publication). The use of analysis of variance increased post-publication, whereas articles using no statistical methods or descriptives only decreased. Approximately one third of articles included covariates in their analysis; inclusion of covariates was consistent from prepublication to post-publication. Findings indicate that over the past 10 years, articles are becoming increasingly rigorous in methodology and reporting practices.