Outliers (typically) cannot cause type I errors in one-sample/paired t-tests

Area of Science:

• Statistics
• Statistical modeling
• Hypothesis testing

Background:

• Outlying data points significantly impact statistical modeling and significance testing.
• Prior research indicates outliers often lead to failing to reject the null hypothesis in one-sample t-tests.
• This study explores the less common scenario where outliers can incorrectly lead to rejecting the null hypothesis.

Purpose of the Study:

• To investigate the conditions under which an outlier can cause the rejection of the null hypothesis in one-sample t-tests.
• To establish mathematical bounds for outliers that increase the t-statistic.
• To assess the practical implications of these findings on Type I error rates.

Main Methods:

• Development of mathematical bounds to determine the maximum size of an outlier that can increase a sample's t-statistic.
• Validation of these bounds using Monte-Carlo simulations.
• Analysis of available data sets to support the theoretical findings.

Main Results:

• Outliers can cause significant results in one-sample t-tests, but only under narrow circumstances.
• Key conditions include the presence of a concordant outlier, a minimum sample size (n ≥ 10), and a small effect size (Cohen's d < 0.5).
• The risk of isolated outliers causing Type I errors is generally low, particularly with small sample sizes.

Conclusions:

• While outliers can lead to Type I errors in one-sample t-tests, the specific conditions required make this a rare event.
• The findings suggest that statistical analyses are robust to outliers in many practical situations.
• Researchers should be aware of these specific conditions when interpreting results from t-tests, especially with larger sample sizes or strong effects.