Extracting data from graphs: A case-study on animal research with implications for meta-analyses

Systematic reviews with meta-analyses are powerful tools that can answer research questions based on data from published studies. Ideally, all relevant data is directly available in the text or tables, but often it is only presented in graphs. In those cases, the data can be extracted from graphs, but this potentially introduces errors. Here, we investigate to what extent the extracted outcome and error values differ from the original data and if these differences could affect the results of a meta-analysis. Six extractors extracted 36 outcome values and corresponding errors from 22 articles. Differences between extractors were compared using overall concordance correlation coefficients (OCCC), differences between the original and extracted data were compared using concordance correlation coefficients (CCC). To test the possible influence on meta-analyses, random-effects meta-analyses on mean difference comparing original and extracted data were performed. The OCCCs and CCCs were high for both outcome values and errors, CCCs were >0.99 for the outcome and >0.92 for errors. The meta-analyses showed that the overall effect on outcome was very small (median: 0.025, interquartile range: 0.016-0.046). Therefore, data extraction from graphs is a good method to harvest data if it is not provided in the text or tables, and the original authors cannot provide the data.