Impact of Inadequate Methods and Data Analysis on Reproducibility

Failure to reproduce results of articles is recognized, but the causes, and therefore solutions, are not. One possibility is that deficits in quality of the work result in varying or inconclusive results. Erythropoiesis-stimulating agents have been used to treat anemia in patients with cancer, but there are concerns that erythropoiesis-stimulating agents might stimulate Epo receptors on tumor cells (Epo receptor-cancer hypothesis). Articles have been published on the topic, but the data and conclusions conflict, making them suitable for examination of a relationship between quality and reproducibility. Comprehensive literature searches were performed, and 280 relevant articles were identified. Numerous conflicts between and within these articles were apparent. The incidence of faults in quality parameters was high, including absence of adequate controls (90% of articles), inadequate validation of reagents and methods (87% of articles), and inadequate or improper statistical methods (84% of articles) with questionable interpretation of the data (81% of articles). This resulted in false-positive/negative data that varied with the reagents and methods used. The low quality of evidence may explain the poor reproducibility of Epo receptor-cancer articles.