Related Experiment Videos
Methodological quality of published systematic reviews of observational epidemiologic studies and their protocol
Judith J M Rijnhart1, Santina Rizzuto1, Ava Rabbers1
1Department of Epidemiology, College of Public Health, University of South Florida, Tampa, FL, USA.
Objective:
The objective of this study was to determine the methodological quality of published systematic reviews of observational studies and their respective protocol registrations.
Introduction:
Historically, methodological guidance for systematic reviews has focused on systematic reviews of randomized controlled trials. Guidance for systematic reviews of observational studies emerged about a decade ago. However, little is known about the methodological quality of systematic reviews of observational studies published to date.
Eligibility Criteria:
Systematic reviews were eligible for inclusion if they i) included observational studies and quasi-experimental studies with human participants, ii) extracted exposure-outcome effect estimates from primary studies, iii) were written in English, and iv) were published in a Q1 Web of Science-indexed or Scopus-indexed epidemiology journal between January 1, 2023, and December 31, 2023 (including advance online publications).
Methods:
This methodological study is a follow-up analysis of a random sample of 50 published systematic reviews of observational epidemiologic studies that were originally identified to examine whether control for third variables was considered as a source of bias or heterogeneity. Where possible, the methods were informed by JBI guidance for umbrella reviews. English-language Q1 epidemiology journals in Scopus and Web of Science were searched to identify records. The search was first performed on January 16, 2024, and repeated on April 1, 2024. For this follow-up analysis, protocol registrations corresponding to the 50 included systematic reviews were retrieved. Data extraction and critical appraisal of the full-text systematic reviews and the protocol registrations were performed independently by 2 reviewers. Methodological quality of the full-text systematic reviews and protocol registrations was assessed using items 1-9 from the JBI Critical Appraisal Checklist for Systematic Reviews and Research Syntheses.
Results:
Of the 50 included systematic reviews, 25 had protocol registrations. The median number of methodological quality items met by the full-text systematic reviews with a protocol registration was 3 (range 1-7) and without a protocol registration was 2 (range 0-5). Item 5, concerning the conduct of critical appraisal, was the only item on which systematic reviews with a protocol registration scored statistically significantly better than systematic reviews without a protocol registration. The median number of methodological quality items met by the protocol registrations was 3 (range 1-6). Agreement in the methodological quality assessment between full-text systematic reviews and protocol registrations was generally high (ie, items were often met or not met across both full-text systematic reviews and protocol registrations).
Conclusions:
The methodological quality of the included systematic reviews of observational studies and their respective protocol registrations was generally low. Further research is needed to identify effective strategies for improving the methodological quality of systematic reviews of observational studies.
Related Concept Videos
Study Designs in Epidemiology
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and case-control studies.
Observational Studies
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One example of...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Bias in Epidemiological Studies
Systematic Error: Methodological and Sampling Errors
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Statistical Methods for Analyzing Epidemiological Data