Related Experiment Video
Updated: Jun 4, 2025
Iterative Development of an Innovative Smartphone-Based Dietary Assessment Tool: Traqq
Published on: March 19, 2021
Reliability and reproducibility of systematic reviews informing the 2020-2025 Dietary Guidelines for Americans: a
Alexandra M Bodnaruc1, Hassan Khan1, Nicole Shaver1
1School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
Background:
Although high-quality nutrition systematic reviews (SRs) are important for clinical decision making, there remains debate on their methodological quality and reporting transparency.
Objectives:
The objective of this study was to assess the reliability and reproducibility of a sample of SRs produced by the Nutrition Evidence Systematic Review (NESR) team to inform the 2020-2025 Dietary Guidelines for Americans (DGAs).
Methods:
We evaluated a sample of 8 SRs from the DGA dietary patterns subcommittee for methodological quality using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool and for reporting transparency using the PRISMA 2020 and PRISMA literature search extension (PRISMA-S) checklists. We assessed the quality and reproducibility of the original search strategy of one selected SR using the Peer Review of Electronic Search Strategies checklist. The reporting transparency of the SR's narrative data synthesis was assessed using the Synthesis Without Meta-Analysis (SWiM) checklist. Interpretation bias was evaluated using existing spin bias classifications in systematic reviews.
Results:
The AMSTAR 2 assessment identified critical methodological weaknesses, and all included SRs were judged to be of critically low quality. Overall, 74% of the PRISMA 2020 checklist items and 63% of the PRISMA-S checklist items were satisfactorily fulfilled. We identified several errors and inconsistencies in the search strategy and could not reproduce searches within a 10% margin of the original results. The SWiM assessment identified concerns regarding the reporting transparency of the narrative data synthesis, but the spin bias assessment revealed no evidence of interpretation bias.
Conclusions:
Several methodological quality and reporting concerns were identified, which could lead to reliability and reproducibility issues should a full reproduction attempt be made. However, additional research is needed to confirm the impact of these findings on conclusions statements and their generalizability across the NESR team SRs. This study was registered in the Open Science Framework (https://osf.io/ns6a9/).
Related Concept Videos
01:29Reliability and Validity
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:20Random and Systematic Errors
01:11Guidelines for Writing Outcome
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
01:52Regression Toward the Mean
01:11Purpose of Health Records I
Here's a breakdown of how health records serve these purposes: