Pitfalls in Study Interpretation

The goal of a randomized or observational study is to develop an unbiased and reliable answer to a therapeutic question. However, there are multiple pitfalls in the reporting and interpretation of data that can compromise our ability to evaluate the pragmatism and the effectiveness of the intervention being studied. Researchers must be conscious of these biases when designing their studies, just as readers must be aware of these potential pitfalls when interpreting results. The purpose of this review is to highlight some of the more common sources of bias in clinical research, including internal and external validity, type 1 and type 2 error, reporting of secondary outcomes, the use of subgroup analyses, and multiple comparisons. This article also discusses potential solutions to these issues, including using the fragility index to understand the robustness of study conclusions, and generating an E value to determine the degree of unmeasured confounding in a study. With an understanding of these pitfalls, readers can critically review scientific literature and ascertain the validity of the conclusions.