Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Meta-analysis: a case study.

Derek C Briggs1

  • 1University of Colorado, Boulder, USA.

Evaluation Review
|February 26, 2005
PubMed
Summary
This summary is machine-generated.

This study questions the value of meta-analysis for social science research. Meta-analysis of SAT coaching studies showed significant prediction errors and problematic statistical inferences, similar to narrative reviews.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Making Inferences About Teacher Observation Scores Over Time.

Educational and psychological measurement·2020
Same author

An introduction to multidimensional measurement using Rasch models.

Journal of applied measurement·2003
Same journal

Brokerage, Gender, and Academic Performance in Interdisciplinary Co-Authorship Networks: A Study of Policy-Related Social Learning Publications.

Evaluation review·2026
Same journal

Peer-led Support Groups for Parents Following Child Removal: A Mixed-Methods Evaluation Study.

Evaluation review·2026
Same journal

Teacher-AI Collaboration to Support Assessment and Feedback: A Case Study in Norwegian Secondary Education.

Evaluation review·2026
Same journal

Green Policies for the Circular Economy and Entrepreneurship: International Evidence.

Evaluation review·2026
Same journal

Transparency, Ethical Framing, and User Agency as Determinants of Trust in AI-Mediated Assessment: Informing the Design of Trustworthy Systems.

Evaluation review·2026
Same journal

No Evidence that Banning the Purchase of Sex Increases Rape: A Replication Study of Ciacci (2024, 2025).

Evaluation review·2026
See all related articles

Area of Science:

  • Social Sciences
  • Educational Research
  • Quantitative Methods

Background:

  • Meta-analysis is a common method for synthesizing quantitative research.
  • Its application in social sciences, particularly for program evaluation, warrants critical examination.
  • Concerns exist regarding the reliability and validity of meta-analytic findings.

Purpose of the Study:

  • To evaluate the utility of meta-analysis for reviewing quantitative research in the social sciences.
  • To assess the predictive accuracy and interpretability of meta-analytic models.
  • To investigate the impact of researcher subjectivity in meta-analytic reviews.

Main Methods:

  • The study employed a meta-analytic model applied to Scholastic Assessment Test (SAT) coaching studies.

Related Experiment Videos

  • Statistical analyses focused on prediction error, regression interpretation, and model-data fit.
  • Comparison was made between meta-analysis and narrative review regarding researcher subjectivity.

    • Main Results:

    • Meta-analytic models for SAT coaching exhibited substantial prediction errors for future studies.
    • Interpretations of meta-analytic regressions and study characteristics were found to be equivocal.
    • The assumptions of the meta-analytic model did not align well with the data from SAT coaching studies, complicating statistical inferences.

    Conclusions:

    • The usefulness of meta-analysis for quantitative social science research is questionable.
    • Meta-analysis does not inherently reduce prediction error or statistical uncertainty compared to other methods.
    • Researcher subjectivity remains a significant concern in meta-analysis, similar to narrative reviews.