Validity and Reliability of GraphClick and DataThief III for Data Extraction

Researchers frequently rely on meta-analyses of prior research studies to efficiently evaluate a broad spectrum of results on a particular topic. In the realm of single-subject experimental designs (SSEDs), meta-analyses have a particular cachet: retaining the rigor of single-subject designs with the added robustness of replication to more fully determine the strength of a given approach or intervention. Until recently, researchers wishing to undertake meta-analytic research themselves have had limited options for synthesizing the intervention effects of a collection of studies. Researchers consistently use two software programs, DataThief III and GraphClick, to conduct meta-analytic work using SSEDs. The purpose of this study was to evaluate and compare the validity and reliability of the results yielded by each of these programs when evaluating the results of multiple research studies on the Good Behavior Game, a classroom-based intervention that has been in practice since 1969. Study findings suggest that both GraphClick and DataThief III provide valid methods of data extraction. In addition, both programs allow for reliable extraction of data between raters and between software programs. Limitations and directions for future research are explored.