The value of lesser-impact-factor surgical journals as a source of negative and inconclusive outcomes reporting

(1) Examine the tendency of peer-reviewed surgical journals to publish positive reports or negative and inconclusive outcome articles as a function of the journals' impact factor (IF). (2) Examine the frequency with which surgical journal editors/publishers adhere to the International Committee of Medical Journal Editors statement on sources of funding and/or conflicts of interest (COI).

Evidence-based medicine is often used as a template for measuring quality of medical care. Clinicians put their faith in peer-reviewed articles as quality-assured and reliable information. However, peer-reviewed literature does not provide balanced access to positive, negative, and inconclusive reports. Funding may also influence the decision to publish certain articles and can thus add to the reported bias in the literature.

Articles from 15 surgical journals comprising 3 separate journal groups based upon 2006 impact factor (IF) rankings were reviewed. All were published in 2007. Manuscripts were classified by 5 independent reviewers as having positive, negative, or inconclusive primary and secondary outcomes and for statements on funding/COI. Positive reports were defined as P < 0.05, null hypothesis rejected; negative reports defined as P < 0.05, null hypothesis accepted; and inconclusive reports defined as P > 0.05. Inter-observer consistency was affirmed. Separate analysis of randomized controlled trials (RCT) was performed to assess for the quality of published positive and negative trials.

We evaluated 2457 published articles. Positive primary outcomes were reported in 67% to 100% of studied articles in selected journals. Negative and inconclusive primary outcomes were less likely to be reported, except for one journal that reported a high of 33% negative articles. Higher-ranked journals published fewer negative and inconclusive studies (5%-7%) than both medium- and lowly-ranked journals (P < 0.0001). The proportion of RCTs published varied, constituting 18% to 21% of articles in the 5 high-ranked journals compared to 6% to 14% in the 5 more lowly ranked journals (P < 0.0001). Reporting of COIs and funding were more frequent in high-IF compared to low-IF journals (P < 0.0001).

Quality rather than outcome should be the measure on which a publication decision is made; commercial bias may further complicate this balance. Lower IF-rated journals may serve a decidedly useful purpose by publishing more negative and inconclusive outcome studies. The practice of focusing disproportionately on the positive outcomes of most studies may result in unbalanced evidence.