Standards and guidelines for observational studies: quality is in the eye of the beholder

Patient care decisions demand high-quality research. To assist those decisions, numerous observational studies are being performed. Are the standards and guidelines to assess observational studies consistent and actionable? What policy considerations should be considered to ensure decision makers can determine if an observational study is of high-quality and valid to inform treatment decisions?

Based on a literature review and input from six experts, we compared and contrasted nine standards/guidelines using 23 methodological elements involved in observational studies (e.g., study protocol, data analysis, and so forth).

Fourteen elements (61%) were addressed by at least seven standards/guidelines; 12 of these elements disagreed in the approach. Nine elements (39%) were addressed by six or fewer standards/guidelines. Ten elements (43%) were not actionable in at least one standard/guideline that addressed the element.

The lack of observational study standard/guideline agreement may contribute to variation in study conduct; disparities in what is considered credible research; and ultimately, what evidence is adopted. A common set of agreed on standards/guidelines for conducting observational studies will benefit funders, researchers, journal editors, and decision makers.