Interpretation of epidemiologic studies very often lacked adequate consideration of confounding

Confounding bias is a most pervasive threat to validity of observational epidemiologic research. We assessed whether authors of observational epidemiologic studies consider confounding bias when interpreting the findings.

We randomly selected 120 cohort or case-control studies published in 2011 and 2012 by the general medical, epidemiologic, and specialty journals with the highest impact factors. We used Web of Science to assess citation metrics through January 2017.

Sixty-eight studies (56.7%, 95% confidence interval: 47.8-65.5%) mentioned "confounding" in the Abstract or Discussion sections, another 20 (16.7%; 10.0-23.3%) alluded to it, and there was no mention or allusion at all in 32 studies (26.7%; 18.8-34.6%). Authors often acknowledged that for specific confounders, there was no adjustment (34 studies; 28.3%) or deem it possible or likely that confounding affected their main findings (29 studies; 24.2%). However, only two studies (1.7%; 0-4.0%) specifically used the words "caution" or "cautious" for the interpretation because of confounding-related reasons and eventually only four studies (3.3%; 0.1-6.5%) had limitations related to confounding or any other bias in their Conclusions. Studies mentioning that the findings were possibly or likely affected by confounding were more frequently cited than studies with a statement that findings were unlikely affected (median 6.3 vs. 4.0 citations per year, P = 0.04).

Many observational studies lack satisfactory discussion of confounding bias. Even when confounding bias is mentioned, authors are typically confident that it is rather irrelevant to their findings and they rarely call for cautious interpretation. More careful acknowledgment of possible impact of confounding is not associated with lower citation impact.