Types of Errors Hiding in Google Scholar Data

Area of Science:

• Bibliometrics and scientometrics
• Information science
• Research evaluation

Background:

• Google Scholar (GS) is a widely used free tool for literature searches and citation analysis.
• Alternative databases like PubMed, PsycINFO, Scopus, and Web of Science are considered more reliable for literature assessment.
• Concerns exist regarding the accuracy of citation data provided by Google Scholar.

Purpose of the Study:

• To examine the accuracy of citation data obtained from Google Scholar.
• To identify and describe errors and miscounts within Google Scholar citation records.
• To assess the potential impact of inaccurate Google Scholar data on research analyses.

Main Methods:

• Retrieved 281 documents that cited two specific works using Publish or Perish (PoP) software.
• Examined each of the 281 retrieved references for accuracy.
• Analyzed error rates, distinguishing between academic and non-academic publications.

Main Results:

• An exceptionally high error rate was found: 279 out of 281 (99.3%) examined references contained at least one error.
• Non-academic documents exhibited a significantly higher error rate compared to academic publications (U=5117.0; P<.001).
• The study identified a significant false-positive issue, particularly relevant in fields like neuroimaging data analysis.

Conclusions:

• Google Scholar data demonstrates a severe lack of accuracy.
• Unverified use of Google Scholar data can introduce substantial biases into researchers' analyses and findings.
• Further research is necessary to fully understand the consequences of using Google Scholar data extracted via tools like Publish or Perish.