Concordance between systematic reviews of randomized controlled trials in assisted reproduction: an overview

Are systematic reviews published within a 3-year period on interventions in ART concordant in their conclusions?

The majority of the systematic reviews published within a 3-year period in the field of assisted reproduction on the same topic had discordant conclusions.

Systematic reviews and meta-analyses have now replaced individual randomized controlled trials (RCTs) at the top of the evidence pyramid. There has been a proliferation of systematic reviews and meta-analyses, many of which suffer from methodological issues and provide varying conclusions.

We assessed nine interventions in women undergoing ART with at least three systematic reviews each, published from January 2015 to December 2017.

The systematic reviews which included RCTs were considered eligible for inclusion. The primary outcome was extent of concordance between systematic reviews on the same topic. Secondary outcomes included assessment of quality of systematic reviews, differences in included studies in meta-analyses covering the same search period, selective reporting and reporting the quality of evidence.

Concordant results and conclusions were found in only one topic, with reviews in the remaining eight topics displaying partial discordance. The AMSTAR grading for the majority of the non-Cochrane reviews was critically low whilst it was categorized as high for all of the Cochrane reviews. For three of the nine topics, none of the included systematic reviews assessed the quality of evidence. We were unable to assess selective reporting as most of the reviews did not have a pre-specified published protocol.

We were limited by the high proportion of reviews lacking a pre-specified protocol, which made it impossible to assess for selective reporting. Furthermore, many reviews did not specify primary and secondary outcomes which made it difficult to assess reporting bias. All the authors of this review were Cochrane review authors which may introduce some assessment bias. The categorization of the review's conclusions as beneficial, harmful or neutral was subjective, depending on the tone and wording of the conclusion section of the review.

The majority of the systematic reviews published within a 3-year period on the same topic in the field of assisted reproduction revealed discordant conclusions and suffered from serious methodological issues, hindering the process of informed healthcare decision-making.

All the authors are Cochrane authors. M.S.K. is an editorial board member of Cochrane Gynaecology and Fertility group. No grant from funding agencies in the public, commercial or not-for-profit sectors was obtained.