Related Experiment Video
Updated: Jul 3, 2026
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
The Prep statistic as a measure of confidence in model fitting
F Gregory Ashby1, Jeffrey B O'Brien
1Department of Psychology, University of California, Santa Barbara, California 93106, USA. ashby@psych.ucsb.edu
Abstract:
In traditional statistical methodology (e.g., the ANOVA), confidence in the observed results is often assessed by computing thep value or the power of the test. In most cases, adding more participants to a study will improve these measures more than will increasing the amount of data collected from each participant. Thus, traditional statistical methods are biased in favor of experiments with large numbers of participants. This article proposes a method for computing confidence in the results of experiments in which data are collected from a few participants over many trials. In such experiments, it is common to fit a series of mathematical models to the resulting data and to conclude that the best-fitting model is superior. The probability of replicating this result (i.e., Prep) is derived for any two nested models. Simulations and empirical applications of this new statistic confirm its utility in studies in which data are collected from a few participants over many trials.
Related Concept Videos
01:19Interpretation of Confidence Intervals
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
01:24Confidence Coefficient
01:21Confidence Intervals
A confidence...
01:19Testing a Claim about Standard Deviation
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
01:24Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:10P-value
P-value stands for the probability value. P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more unlikely...