Related Experiment Video
Updated: Nov 1, 2025
Author Spotlight: Biological Standardization to Ensure Reproducibility and Harmonization in Research
Published on: August 4, 2023
Methodological standards for conducting and reporting meta-analyses: Ensuring the replicability of meta-analyses of
Aline F Bonetti1, Fernanda S Tonin1, Rosa C Lucchetta1
1Pharmaceutical Sciences Postgraduate Program, Federal University of Paraná, Curitiba, Brazil.
Background:
Meta-analyses of clinical pharmacy services are frequently criticized for restricted data transparency and reproducibility.
Objectives:
To describe the methodological characteristics of meta-analyses of pharmacist-led medication reviews, to identify the elements that limit their replicability and robustness, and to propose recommendations for an appropriate conduction and reporting.
Methods:
A meta-research study was conducted. Systematic searches of the PubMed, Scopus, and Web of Science databases were performed to identify meta-analyses of pharmacist services. Meta-analyses assessing the effect of pharmacist-led medication reviews were selected for data extraction, analysis and replication. Two replication exercises were performed for the two most common outcomes: (i) considering the data provided by authors to construct the meta-analysis and (ii) considering the raw data available in the primary studies included. Prediction intervals (PI), fragility index (FI), and number needed to treat (NNT) were also calculated for each replicated meta-analysis.
Results:
Nine studies reporting meta-analyses about pharmacist-led medication review were found comprising 30 different outcomes. Eleven meta-analyses, including six for hospital admission and five for mortality, were replicated. In five meta-analyses, the pooled effect sizes of the replicated meta-analyses differed from the original ones. Only four meta-analyses mentioned the statistical method used. Other meta-analytic parameters (e.g., q-value, tau2) were omitted in all studies. In nine meta-analyses, the data from primary studies had been incorrectly extracted for at least one variable. The PI demonstrated that the uncertainty intervals of the effect sizes were always underestimated by the authors. NNTs showed wide intervals, ranging from benefit to harm, in almost all meta-analyses. Nine recommendations to facilitate the replication of a meta-analysis were proposed: providing all original data needed to build the analysis; informing about the imputed data or data obtained from different sources; performing sensitivity analyses for imputed or unpublished data; inform about all the statistical methods used; providing all statistical results; and reporting the PI, FI and NNT.
Conclusion:
Errors in data extraction and poor reporting of meta-analytic parameters are common in the pharmacy literature. We proposed nine recommendations to enhance data reproducibility and interpretability. Journal editors and peer reviewers should ensure that authors strictly comply with minimum standards for conduction and reporting of meta-analyses.
Insights
Meta-analyses in pharmacy are often unreproducible due to data errors. This study proposes nine recommendations to improve data transparency and reporting standards for pharmacist-led medication reviews.
Area of Science:
- Pharmacoeconomics and Health Outcomes Research
- Evidence Synthesis and Meta-Research
Background:
- Meta-analyses of clinical pharmacy services face criticism regarding limited data transparency and reproducibility.
- Replicability issues hinder the robustness and reliability of findings in systematic reviews within pharmacy.
Purpose of the Study:
- To characterize the methodology of meta-analyses on pharmacist-led medication reviews.
- To identify factors limiting the replicability and robustness of these meta-analyses.
- To propose recommendations for improving the conduct and reporting of such studies.
Main Methods:
- A meta-research study involving systematic database searches (PubMed, Scopus, Web of Science).
- Selection of meta-analyses on pharmacist-led medication reviews for data extraction, analysis, and replication.
- Two replication exercises were performed, assessing author-provided data and raw primary study data, calculating prediction intervals (PI), fragility index (FI), and number needed to treat (NNT).
Main Results:
- Eleven meta-analyses were replicated, with five showing discrepancies in pooled effect sizes compared to original findings.
- Significant reporting deficiencies were noted, including omission of statistical methods and meta-analytic parameters, and incorrect data extraction in nine meta-analyses.
- Prediction intervals indicated underestimated uncertainty, and number needed to treat values showed wide ranges from benefit to harm.
Conclusions:
- Errors in data extraction and inadequate reporting of meta-analytic parameters are prevalent in pharmacy literature.
- Nine recommendations are proposed to enhance data reproducibility and interpretability.
- Journal editors and peer reviewers must enforce minimum standards for conducting and reporting meta-analyses.
Related Concept Videos
01:19Pharmacovigilance
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:26Therapeutic Drug Monitoring: Drug Analysis Methods
02:07Blind Procedures
01:12Analysis of Population Pharmacokinetic Data
01:27Pharmaceutical Alternatives: Polymorphic Form-Related and Particle Size-Related Therapeutic Nonequivalence