Biomedical journals lack a consistent method to detect outcome reporting bias: a cross-sectional analysis

Area of Science:

• Medical Research Integrity
• Scholarly Publishing
• Biostatistics

Background:

• Outcome reporting bias (ORB) is increasingly recognized as a significant factor distorting data in clinical trials and systematic reviews.
• Despite its impact, current methods employed by biomedical journals to detect ORB remain largely unknown.

Purpose of the Study:

• To investigate the current practices of top-ranked biomedical journals in detecting and managing outcome reporting bias.
• To assess the prevalence of ORB detection methods and the perceived frequency of ORB within the journal review process.

Main Methods:

• A cross-sectional analysis was conducted using standardized questionnaires.
• Data were collected via email or telephone from the top 30 biomedical journals based on 2012 impact factor (excluding one journal, n=29).

Main Results:

• Only 14 of 29 journals fully responded; 5 (36%) reported specific methods for ORB detection, while 9 (64%) did not.
• Journals' perceptions of ORB prevalence varied, with 4 (29%) finding it common and 7 (50%) finding it uncommon or never detected.
• The majority of responding journals (72%) were unwilling to share discrepancies publicly, though 29% described thorough detection methods.

Conclusions:

• Many journals appear to lack adequate methods for detecting ORB, suggesting its actual prevalence may be underestimated.
• This gap in detection may lead to an overestimation of treatment effects and unclear risks in published literature.
• While some journals employ comprehensive ORB detection strategies, overall improvements are necessary at the biomedical journal level.