Primary Study Quality in Psychological Meta-Analyses: An Empirical Assessment of Recent Practice

Area of Science:

• Psychological Sciences
• Research Methodology

Background:

• Meta-analytic research is increasingly prevalent in psychology.
• Clear reporting of primary study quality is vital for meta-analysis interpretability and reproducibility.
• Existing notions of study quality can vary significantly across different research domains.

Purpose of the Study:

• To examine the current state of reporting practices for primary study quality in published psychological meta-analyses.
• To understand the reporting decisions and motivations of authors regarding primary study quality assessments.
• To identify reasons for poor adherence to reporting standards in psychological meta-analyses.

Main Methods:

• A systematic review of 382 published psychological meta-analyses.
• Analysis of reporting practices related to primary study quality.
• Investigation into authors' reporting decisions and motivations.

Main Results:

• Adherence to reporting standards for primary study quality in meta-analyses is consistently poor.
• Reporting practices within the discipline are generally inconsistent.
• Potential reasons include non-conduct of quality assessments, lack of awareness of standards, or conflation with other meta-analytic procedures.

Conclusions:

• Suboptimal reporting of primary study quality in psychological meta-analyses poses a significant challenge.
• Inconsistent practices and potential lack of awareness hinder the reliability of meta-analytic findings.
• Improving reporting standards is crucial for enhancing the interpretability and reproducibility of psychological research.