Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Censoring Survival Data01:09

Censoring Survival Data

654
Survival analysis is a statistical method used to analyze time-to-event data, often employed in fields such as medicine, engineering, and social sciences. One of the key challenges in survival analysis is dealing with incomplete data, a phenomenon known as "censoring." Censoring occurs when the event of interest (such as death, relapse, or system failure) has not occurred for some individuals by the end of the study period or is otherwise unobservable, and it might have many different...
654
Assumptions of Survival Analysis01:15

Assumptions of Survival Analysis

489
Survival models analyze the time until one or more events occur, such as death in biological organisms or failure in mechanical systems. These models are widely used across fields like medicine, biology, engineering, and public health to study time-to-event phenomena. To ensure accurate results, survival analysis relies on key assumptions and careful study design.
489
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

7.5K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
7.5K
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

4.6K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
4.6K
Statistical Hypothesis Testing01:16

Statistical Hypothesis Testing

7.1K
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
7.1K
Outliers and Influential Points01:08

Outliers and Influential Points

6.7K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
6.7K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A Post-Processing Fairness Mitigation Method for Medical Prediction Models.

IEEE journal of biomedical and health informatics·2026
Same author

Reply to Sanchéz et al.: Multiplicity does not protect privacy.

Proceedings of the National Academy of Sciences of the United States of America·2023
Same author

Confidence-ranked reconstruction of census microdata from published statistics.

Proceedings of the National Academy of Sciences of the United States of America·2023
Same author

Universal adaptability: Target-independent inference that competes with propensity scoring.

Proceedings of the National Academy of Sciences of the United States of America·2022
Same author

Algorithmic amplification of politics on Twitter.

Proceedings of the National Academy of Sciences of the United States of America·2021
Same author

A Unified Strategy for the Asymmetric Synthesis of Highly Substituted 1,2-Amino Alcohols Leading to Highly Substituted Bisoxazoline Ligands.

The Journal of organic chemistry·2021

Related Experiment Video

Updated: Apr 5, 2026

A Cost Effective and Adaptable Scratch Migration Assay
08:59

A Cost Effective and Adaptable Scratch Migration Assay

Published on: June 30, 2020

6.4K

STATISTICS. The reusable holdout: Preserving validity in adaptive data analysis.

Cynthia Dwork1, Vitaly Feldman2, Moritz Hardt3

  • 1Microsoft Research, Mountain View, CA 94043, USA. dwork@microsoft.com vitaly@post.harvard.edu m@mrtz.org toni@cs.toronto.edu omer.reingold@gmail.com aaroth@cis.upenn.edu.

Science (New York, N.Y.)
|August 8, 2015
PubMed
Summary
This summary is machine-generated.

Researchers developed a new statistical method to prevent false discoveries from adaptive data analysis. This approach uses privacy-preserving techniques to safely validate findings from exploratory data science, improving research reliability.

More Related Videos

The Attentional Set Shifting Task: A Measure of Cognitive Flexibility in Mice
09:15

The Attentional Set Shifting Task: A Measure of Cognitive Flexibility in Mice

Published on: February 4, 2015

28.8K
Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
06:48

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Published on: June 25, 2019

9.9K

Related Experiment Videos

Last Updated: Apr 5, 2026

A Cost Effective and Adaptable Scratch Migration Assay
08:59

A Cost Effective and Adaptable Scratch Migration Assay

Published on: June 30, 2020

6.4K
The Attentional Set Shifting Task: A Measure of Cognitive Flexibility in Mice
09:15

The Attentional Set Shifting Task: A Measure of Cognitive Flexibility in Mice

Published on: February 4, 2015

28.8K
Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
06:48

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Published on: June 25, 2019

9.9K

Area of Science:

  • Statistical methodology
  • Data science
  • Scientific research integrity

Background:

  • Misapplication of statistical data analysis frequently leads to spurious discoveries in scientific research.
  • Current methods for validating data inferences rely on pre-defined, fixed analytical procedures.
  • Real-world data analysis is inherently adaptive, evolving through data exploration and prior results.

Purpose of the Study:

  • To introduce a novel approach for validating inferences from adaptive data analysis.
  • To address the challenges posed by the adaptive nature of modern data exploration.
  • To enhance the reliability of scientific discoveries derived from complex datasets.

Main Methods:

  • Developed a new statistical validation framework inspired by privacy-preserving data analysis techniques.
  • Demonstrated the application of this framework using a holdout dataset.
  • Showcased the safe, repeated reuse of a holdout set for validating adaptively chosen analyses.

Main Results:

  • The proposed method effectively addresses the challenges of adaptivity in statistical analysis.
  • A holdout dataset can be safely reused multiple times for validation purposes.
  • This approach enhances the trustworthiness of findings generated through exploratory data analysis.

Conclusions:

  • A new statistical approach enables reliable validation of adaptive data analyses.
  • Insights from privacy-preserving methods offer solutions for ensuring research integrity.
  • This work provides a practical method to mitigate spurious discoveries in scientific research.