Multiple Confidence Intervals and Surprisal Intervals to Avoid Significance Fallacy.

Area of Science:

• Biostatistics
• Medical Statistics
• Statistical Inference

Background:

• Overconfidence in statistical results, particularly in medicine, stems from improper practices and historical biases related to statistical significance.
• Misinterpretations include dichotomizing results (significant vs. not significant), conflating different statistical approaches, and the magnitude/nullification fallacies.
• These issues distort the purpose of statistical investigations and hinder their ability to inform public health and other scientific fields.

Discussion:

• The international statistical community has proposed alternatives to address these issues, but misuses of statistical significance persist.
• This paper advocates for the use of multiple confidence (or compatibility) intervals to tackle these problems directly.
• An extension, the 'surprisal interval' (S-interval), is introduced within the framework of statistical surprisal.

Key Insights:

• Multiple compatibility intervals offer a more robust approach to statistical interpretation than traditional significance testing.
• The proposed surprisal interval provides a novel way to understand statistical surprise, drawing parallels to coin flipping.
• This method allows for a complete departure from the problematic concepts of statistical significance and confidence.

Outlook:

• Adoption of compatibility and surprisal intervals could lead to more accurate statistical interpretations in medicine and science.
• Improved statistical practices can enhance the reliability of public health decisions and scientific research.
• Further exploration of statistical surprisal and its applications is warranted.