Case study in major quotation errors: a critical commentary on the Newcastle-Ottawa scale

The Newcastle-Ottawa scale (NOS) is one of many scales used to judge the quality of observational studies in systematic reviews. It was criticized for its arbitrary definitions of quality items in a commentary in 2010 in this journal. That commentary was cited 1,250 times through December 2016. We examined the citation history of this commentary in a random sample of 100 full papers citing it, according to the Web of Science. Of these, 96 were systematic reviews, none of which quoted the commentary directly. All but 2 of the 96 indirect quotations (98%) portrayed the commentary as supporting use of the NOS in systematic reviews when, in fact, the opposite was the case. It appears that the vast majority of systematic review authors who cited this commentary did not read it. Journal reviewers and editors did not recognize and correct these major quotation errors. Authors should read each source they cite to make sure their direct and indirect quotations are accurate. Reviewers and editors should do a better job of checking citations and quotations for accuracy. It might help somewhat for commentaries to include abstracts, so that the basic content can be conveyed by PubMed and other bibliographic resources.