Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Statistical Analysis: Overview01:11

Statistical Analysis: Overview

6.1K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.1K
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

175
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
175
Regression Toward the Mean01:52

Regression Toward the Mean

6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Statistical Significance01:50

Statistical Significance

20.1K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.1K
Data Validation01:15

Data Validation

142
Method validation is a crucial process in analytical chemistry designed to confirm that a given method consistently produces reliable and high-quality results. This process is essential when a method is applied to different sample matrices or when procedural modifications are made, ensuring that the results meet acceptable standards across various applications.
Key parameters for method validation include:
142
Reliability and Validity01:29

Reliability and Validity

12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Core principles of responsible generative AI usage in research.

AI and ethics·2025
Same author

Bayes factors for two-group comparisons in Cox regression with an application for reverse-engineering raw data from summary statistics.

Journal of applied statistics·2025
Same author

When theory and therapy part ways-A scoping review of the science-to-practice gap.

Psychotherapy research : journal of the Society for Psychotherapy Research·2025
Same author

To interact or not to interact: The pros and cons of including interactions in linear regression models.

Behavior research methods·2025
Same author

How can we make sound replication decisions?

Proceedings of the National Academy of Sciences of the United States of America·2025
Same author

Practical implications of equating equivalence tests: Reply to Campbell and Gustafson (2022).

Psychological methods·2024
Same journal

Invaders taking over-Mollusc faunal change in volcanic barrier lakes of the Albertine Rift biodiversity hotspot.

PloS one·2026
Same journal

AI-driven molecular diversification and ligand-based optimization of macitentan derivatives targeting VEGFR1 and endothelin signaling pathways.

PloS one·2026
Same journal

Performance patterns and records in the world aquatics masters championships: Where do the most frequently represented nations among the top-ten masters swimmers come from?

PloS one·2026
Same journal

Modeling diurnal Temperature-Rainfall relationships under multicollinearity using PLS-SEM: A case study of Ghana.

PloS one·2026
Same journal

Organizational culture, social capital, and emergency capacity in primary healthcare institutions: A cross-sectional structural equation modeling study comparing ordinary and older communities.

PloS one·2026
Same journal

Impact of kidney function on the metabolome in the general population.

PloS one·2026
See all related articles

Related Experiment Video

Updated: Jun 5, 2025

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
08:36

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Published on: April 19, 2024

436

Evaluating meta-analysis as a replication success measure.

Jasmine Muradchanian1, Rink Hoekstra1, Henk Kiers1

  • 1Behavioural and Social Sciences, University of Groningen, Groningen, the Netherlands.

Plos One
|December 11, 2024
PubMed
Summary
This summary is machine-generated.

Meta-analysis performs poorly as a replication success metric in social sciences. It often incorrectly concludes success, regardless of replication results, due to original study significance.

More Related Videos

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
06:26

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI

Published on: November 27, 2019

70.0K
The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan
11:58

The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan

Published on: June 29, 2018

9.4K

Related Experiment Videos

Last Updated: Jun 5, 2025

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
08:36

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Published on: April 19, 2024

436
Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
06:26

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI

Published on: November 27, 2019

70.0K
The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan
11:58

The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan

Published on: June 29, 2018

9.4K

Area of Science:

  • Social and behavioral sciences
  • Research methodology
  • Statistical analysis

Background:

  • Replication is crucial in social and behavioral sciences.
  • Meta-analysis is one method to assess replication success.
  • Previous methods for qualifying replication success exist.

Purpose of the Study:

  • To evaluate meta-analysis as a metric for replication success.
  • To investigate how meta-analysis functions in qualifying replication outcomes.
  • To assess the accuracy of meta-analysis predictions of replication success.

Main Methods:

  • Utilized original and replication studies from large-scale replication projects.
  • Calculated the probability of replication success using meta-analysis.
  • Assessed prediction accuracy with adjusted Brier scores after replication results were known.

Main Results:

  • Meta-analysis performed poorly as a replication success metric.
  • Conclusions of success were often reached irrespective of actual replication study findings.
  • Meta-analysis showed a high probability of detecting a non-zero effect even when the true effect was zero.

Conclusions:

  • Meta-analysis is not a reliable metric for replication success.
  • Original study significance heavily influences meta-analysis outcomes.
  • Fundamental issues exist with using meta-analysis to judge replication success.