Challenges for assessing replicability in preclinical cancer biology

Area of Science:

• Biomedical Research
• Cancer Biology
• Scientific Reproducibility

Background:

• The Reproducibility Project: Cancer Biology aimed to assess the replicability of preclinical cancer research.
• The project initially planned to repeat 193 experiments from 53 high-impact publications.

Purpose of the Study:

• To identify and report the barriers and challenges encountered during the replication of cancer biology experiments.
• To evaluate the feasibility of repeating published preclinical cancer research.

Main Methods:

• Attempted to repeat 193 experiments from 53 cancer biology papers, with peer-reviewed protocols and data analysis plans.
• Documented issues related to statistical reporting, experimental detail, author communication, and protocol modifications.

Main Results:

• Only 50 experiments from 23 papers were successfully repeated due to significant challenges.
• Key statistics were publicly available for only 4 experiments; author assistance was inconsistent (helpful for 41%, unhelpful/unresponsive for 32%).
• 67% of protocols required modifications, with only 41% of those modifications being implementable.

Conclusions:

• Replicating preclinical cancer biology research faces substantial obstacles, including lack of detailed methodology and statistical data.
• Inconsistent author support and necessary protocol adaptations further impede replication efforts.
• The difficulties encountered raise fundamental questions about the credibility and reliability of published scientific findings in cancer biology.