P>0.05 Is Good: The NORD-h Protocol for Several Hypothesis Analysis Based on Known Risks, Costs, and Benefits.

Area of Science:

• Medical Statistics
• Epidemiology
• Public Health Research

Background:

• Statistical testing in medicine is prone to common misunderstandings and fallacies, such as nullism and dichotomania.
• These misinterpretations can lead to significant errors in clinical and epidemiological research, potentially impacting patient safety and health outcomes.
• A prevalent error is the 'fallacy of high significance,' prioritizing low p-values even when high p-values are desirable for certain hypotheses.

Purpose of the Study:

• To address the fallacy of high significance in statistical interpretation.
• To propose a novel statistical method that moves beyond null hypothesis testing.
• To formalize interval hypotheses incorporating stakeholder-specific costs, risks, and benefits.

Main Methods:

• Developing a method to assess statistical surprise by comparing experimental results against predictions from target hypotheses.
• Introducing the NORD-h protocol for formalizing interval hypotheses using prior information on costs, risks, and benefits.
• Utilizing incompatibility graphs (surprisal graphs) to analyze statistical relationships.

Main Results:

• The proposed method offers an alternative to traditional null hypothesis significance testing.
• It allows for the evaluation of statistical surprise relative to predefined target assumptions.
• The NORD-h protocol provides a framework for incorporating practical considerations into statistical inference.

Conclusions:

• The novel statistical approach can mitigate interpretive and cognitive errors in medical research.
• This method has the potential to enhance the reliability and validity of evidence in public health.
• A descriptive, (quasi) unconditional statistical approach is essential for drawing robust conclusions, considering all possibilities and study limitations.