Do multiple experimenters improve the reproducibility of animal studies?

Area of Science:

• Animal research
• Reproducibility in science

Background:

• The reproducibility crisis questions scientific research credibility.
• Standardizing experimental conditions in animal studies may hinder reproducibility.
• Systematic heterogenisation is proposed to enhance reproducibility but requires real-world testing.

Purpose of the Study:

• To evaluate if heterogenisation of experimental conditions, using multiple experimenters, enhances research reproducibility compared to standardized conditions with a single experimenter.
• To assess the impact of experimenter variation versus laboratory variation on reproducibility in animal studies.

Main Methods:

• Replicated the same animal experiment in three independent laboratories.
• Each laboratory used both a standardized design (one experimenter) and a heterogenised design (three experimenters).
• Utilized variance component analysis to identify sources of variation.

Main Results:

• Heterogenisation with multiple experimenters did not improve reproducibility across laboratories.
• Laboratory variation and residual variation were more significant sources of variance than experimenter variation.
• Residual variation accounted for 41%-72% of the total observed variance in most outcome measures.

Conclusions:

• The strategy of using multiple experimenters for heterogenisation did not yield the expected improvement in reproducibility.
• Findings support systematically including biological variation in animal studies rather than eliminating it.
• Further research is needed to identify effective strategies for improving reproducibility in animal research.