Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Variation01:19

Variation

An important characteristic of any set of data is the variation in the data. In some data sets, the data values are concentrated closely near the mean; in other data sets, the data values are more widely spread out from the mean. The most common measure of variation, or spread, is the standard deviation, which is the square root of variance.
When independent and dependent variables are plotted on a scatter plot, the slope of a line is a value that describes the rate of change between the two...
Comparing Experimental Results: Student's t-Test01:09

Comparing Experimental Results: Student's t-Test

The t-test is a statistical method used to compare the sample mean with a population mean or compare two means from two data sets. The test statistic is calculated from the standard deviation, mean, and number of measurements in the data set at a selected confidence interval and then compared to a table of critical values at this confidence level. If the test statistic is smaller than the critical value, the null hypothesis is accepted. In this case, we state that the difference between the...
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
Sign Test for Matched Pairs01:17

Sign Test for Matched Pairs

The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in value between...
Bonferroni Test01:10

Bonferroni Test

The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
Prediction Intervals01:03

Prediction Intervals

The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y. 
The...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Statescope: an integrative deconvolution framework for discovering cell states in tumors.

Nature communications·2026
Same author

Triage Performance of FAM19A4/miR124-2 Methylation: 18-Month Follow-Up Results From a Prospective Observational Study Within the Dutch Primary HPV-Based Cervical Screening Program.

International journal of cancer·2026
Same author

Activity and Physiological Stress Within 90 Days After Minimally Invasive and Open Pancreatoduodenectomy: A Predefined Analysis of the DIPLOMA-2 Randomized Clinical Trial.

JAMA surgery·2026
Same author

Recurrence in High-grade Vulvar Intraepithelial Neoplasia: A Comprehensive Cohort Analysis.

Journal of lower genital tract disease·2026
Same author

Prognostic value of peri-operative circulating tumour DNA levels estimated by cell-free DNA methylation in patients with resectable colorectal liver metastases.

EBioMedicine·2026
Same author

A Comparison of Methods for Modeling Multistate Cancer Progression Using Screening Data with Censoring after Intervention.

Medical decision making : an international journal of the Society for Medical Decision Making·2026

Related Experiment Video

Updated: Jun 23, 2026

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
14:14

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups

Published on: May 13, 2022

Testing the prediction error difference between 2 predictors.

Mark A van de Wiel1, Johannes Berkhof, Wessel N van Wieringen

  • 1Department of Epidemiology and Biostatistics, VU University Medical Center, PO Box 7057, 1007 MB Amsterdam, The Netherlands. mark.vdwiel@vumc.nl

Biostatistics (Oxford, England)
|April 22, 2009
PubMed
Summary
This summary is machine-generated.

We developed a new statistical framework to compare prediction methods, using paired residuals and sample splitting for robust error difference inference. This approach enhances statistical power for analyzing complex biological data.

More Related Videos

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model
07:13

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

Related Experiment Videos

Last Updated: Jun 23, 2026

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
14:14

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups

Published on: May 13, 2022

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model
07:13

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

Area of Science:

  • Statistical methodology
  • Bioinformatics
  • Genomics

Background:

  • Comparing prediction procedures requires robust statistical inference.
  • Existing methods may not adequately handle differences in covariates or procedural aspects.
  • Genomic data analysis often involves complex prediction tasks.

Purpose of the Study:

  • To develop a flexible inference framework for comparing any two prediction procedures.
  • To provide theoretical guarantees on type I error rates.
  • To demonstrate the framework's utility and power in genomic survival data analysis.

Main Methods:

  • Utilizing sample splitting for training and testing on the same dataset.
  • Generating paired residuals from two prediction procedures for each sample split.
  • Applying a signed-rank test to paired residuals and proposing median p-value and mean inverse normal transformed p-value as summary statistics.

Main Results:

  • Theoretical bounds on the overall type I error rate are established.
  • Simulation studies confirm type I error control and superior power compared to a one-split approach.
  • The framework successfully compared LASSO and ridge regression and evaluated marker utility in genomic survival data.

Conclusions:

  • The proposed inference framework offers a powerful and flexible tool for comparing diverse prediction procedures.
  • It provides reliable error rate control and enhanced statistical power, particularly for complex datasets.
  • The method is broadly applicable to various prediction paradigms in bioinformatics and beyond.