Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Bias01:22

Bias

6.5K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
6.5K
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

2.9K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.9K
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

6.6K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.6K
Significance Testing: Overview01:04

Significance Testing: Overview

7.9K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
7.9K
Statistical Significance01:50

Statistical Significance

20.6K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.6K
Bonferroni Test01:10

Bonferroni Test

2.9K
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
2.9K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Reducing the biases of the conventional meta-analysis of correlations.

Research synthesis methods·2026
Same author

Robust Bayesian meta-regression: Model-averaged moderation analysis in the presence of publication bias.

Psychological methods·2025
Same author

Correcting bias in the meta-analysis of correlations.

Psychological methods·2024
Same author

Exploring open science practices in behavioural public policy research.

Royal Society open science·2024
Same author

Estimating the extent of selective reporting: An application to economics.

Research synthesis methods·2024
Same author

Meta-analyses of partial correlations are biased: Detection and solutions.

Research synthesis methods·2024

Related Experiment Video

Updated: Oct 31, 2025

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.7K

Detecting publication selection bias through excess statistical significance.

T D Stanley1, Hristos Doucouliagos2, John P A Ioannidis3,4

  • 1School of Business and Law, Deakin University, Burwood, Victoria, Australia.

Research Synthesis Methods
|July 1, 2021
PubMed
Summary

We developed new methods to detect publication bias using excess statistical significance (ESS). These tests, including PSST and TESS, are more effective than existing Egger and 3PSM methods in simulations.

Keywords:
excess statistical significancemeta-analysispublication selection biasstatistical power

More Related Videos

Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data
04:57

Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data

Published on: May 16, 2022

16.5K
Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
06:26

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI

Published on: November 27, 2019

74.7K

Related Experiment Videos

Last Updated: Oct 31, 2025

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.7K
Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data
04:57

Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data

Published on: May 16, 2022

16.5K
Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
06:26

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI

Published on: November 27, 2019

74.7K

Area of Science:

  • Biostatistics
  • Medical Research Methodology
  • Meta-Analysis

Background:

  • Publication selection bias is a significant issue in scientific literature.
  • Existing methods like Egger's test and the three-parameter selection model (3PSM) have limitations in detecting this bias.
  • Excess statistical significance (ESS) is a potential indicator of publication bias.

Purpose of the Study:

  • To introduce and evaluate three novel statistical tests for publication selection bias.
  • To explicitly incorporate heterogeneity into the assessment of ESS.
  • To compare the performance of the new ESS tests against conventional methods.

Main Methods:

  • Developed three ESS-based tests: Proportion of Statistical Significance Test (PSST), Test of Excess Statistical Significance (TESS), and a combined TESSPSST.
  • Calculated expected proportions of significant findings, accounting for study SE and meta-analysis estimates of true-effect distribution.
  • Conducted simulations to assess test performance and compared them to Egger's test and 3PSM.

Main Results:

  • The proposed ESS tests, particularly TESS and TESSPSST, demonstrated superior performance in simulations compared to Egger's test and 3PSM.
  • The new methods effectively identified publication selection bias by considering heterogeneity.
  • Simulations indicated that the ESS tests are robust in detecting bias.

Conclusions:

  • The novel ESS-based tests offer a more sensitive and accurate approach to detecting publication selection bias.
  • Incorporating heterogeneity explicitly improves the detection of bias.
  • These methods provide valuable tools for researchers and meta-analysts to ensure the reliability of scientific findings.