Related Experiment Video
Updated: Jun 11, 2025
07:35
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
Published on: October 11, 2018
7.4K
A cautionary note against selective applications of the Bayes factor
Marcel R Schreiner1, Wilfried Kunde1
1Department of Psychology, Julius-Maximilians-Universitat Wurzburg.
Journal of Experimental Psychology. General
|October 7, 2024
Summary
Selective use of Bayesian analysis after frequentist tests can overestimate evidence for the null hypothesis. This approach, combining frequentist and Bayesian methods, introduces bias when a true effect exists, recommending consistent use of either approach.
Area of Science:
- Statistics
- Psychological Research Methods
Background:
- Bayes factor analysis is gaining popularity for its ability to support the null hypothesis, unlike traditional frequentist methods.
- A common strategy involves using frequentist analyses first, followed by Bayesian analyses for theoretically interesting null effects.
Purpose of the Study:
- To investigate whether selectively applying Bayesian analyses to nonsignificant frequentist results introduces bias.
- To assess the impact of this combined analytical strategy on the evidence for null hypotheses.
Main Methods:
- Two simulation studies were conducted to examine the proposed analytical approach.
- The simulations focused on scenarios where a true population effect was present.
Main Results:
- The selective application of Bayesian analyses severely overestimated evidence favoring the null hypothesis when a true population effect existed.
- Using more informative priors in Bayesian analyses could attenuate this bias.
Conclusions:
- The practice of selectively combining frequentist and Bayesian analyses is not recommended due to introduced bias.
- Researchers should consistently employ either frequentist or Bayesian analyses for robust and unbiased results.
More Related Videos
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
123
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
123
01:10Bonferroni Test
2.7K
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
2.7K
01:24Testing a Claim about Population Proportion
3.3K
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
3.3K
01:04Significance Testing: Overview
3.3K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
3.3K
01:16Critical Region, Critical Values and Significance Level
11.8K
The critical region, critical value, and significance level are interdependent concepts crucial in hypothesis testing.
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
11.8K
01:22Bias
3.8K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
3.8K