Related Experiment Videos
Reviewer bias against the unconventional? A randomized double-blind study of peer review
Objective:
To test the hypothesis that there is a reviewer bias against publication of a test of an unconventional drug.
Design:
Randomized, controlled, double-blind study of peer review.
Participants:
Convenience sample of 291 medical doctors from a wide variety of specialties drawn from a list of conference participants.
Methods:
Reviewers were randomly assigned to receive one of two versions of a manuscript. Version M related to an in-vitro experiment on a mainstream drug (Metoprolol). The otherwise identical version V used a highly unconventional drug (beef spleen cell extract) for the same experiment. Reviewers were asked to complete a standardised evaluation sheet including visual analogue scales (VASs) on a set of predefined quality criteria. All participants were debriefed after completion of the study.
Results:
The response rate was 61%. There were no significant differences in VAS ratings between the two versions of the manuscript. Ratings covered the entire range of the VASs.
Conclusion:
In the present setting, there was no evidence for a reviewer-bias against testing an unconventional drug. The low inter-rater reliability, however, suggested inadequate validity of peer review.