Reverse fragility index shows high fragility in Cochrane meta-analyses with P values between 0.05 and 0.20: a...

To quantify fragility of statistically nonsignificant meta-analysis results with P values between 0.05 and 0.20 using the reverse fragility index.

This meta-epidemiological study examined Cochrane systematic reviews published in 2025. We calculated the reverse fragility index (minimum number of outcome event changes needed to convert a nonsignificant result to statistically significant) and reverse fragility quotient. Results represent medians (Q1-Q3) with 95% confidence interval (CI) from 10,000 bootstrap iterations.

We analyzed 280 nonsignificant pooled effect estimates from 81 Cochrane systematic reviews. Meta-analyses included a median of five studies and 1002 participants with event risks of 11.7% and 12.6% in the intervention and control groups, respectively. The median reverse fragility index was 3 (95% CI, 2-3; Q1-Q3, 2-5), indicating that modifying three events among the primary studies included in a meta-analysis could change the status of nonsignificant results. The median reverse fragility quotient was 2.7 per 1000 participants (95% CI, 2.2-3.3; Q1-Q3, 1.0-5.8). Meta-analyses with P values 0.05-0.10 were more fragile than those with P values 0.10-0.20. Event changes did not substantially increase heterogeneity.

Meta-analyses with P values between 0.05 and 0.20 showed statistical fragility. Reverse fragility metrics provide quantifiable evidence for nonsignificant results that may have a signal of clinical relevance and challenge their misinterpretation as robust null findings.

When researchers combine numerical results from multiple studies (called meta-analyses), they often find that results are not "statistically significant," meaning that they are more likely due to chance or error. These results are commonly misinterpreted using phrases such as "there is no difference" between the two groups, which can mislead readers about what the evidence shows. We examined 280 meta-analyses from Cochrane systematic reviews to test the stability of these results. We used a method called the "reverse fragility index," which counts how many events would need to change from not occurring to occurring or vice versa to shift a statistically nonsignificant result into statistically significant. We found that these statistically nonsignificant results are fragile. In more than half of the meta-analyses, changing three events or fewer was enough to make a nonsignificant result significant. This represents almost three event changes per 1000 people. The findings show that meta-analyses with P values between 0.05 and 0.20 should not be treated as strong evidence that an intervention does not work or does not cause harm. Instead, these results are fragile. Researchers should report how fragile their nonsignificant results are to help clinicians and guideline developers better understand how much they can trust the evidence when making clinical decisions.