Using Bayes factors to evaluate evidence for no effect: examples from the SIPS project.

Area of Science:

• Statistics
• Public Health
• Clinical Trials

Background:

• Significance testing can lead to inappropriate conclusions when results are non-significant.
• The Screening and Intervention Programme for Sensible drinking (SIPS) project provides a case study for re-evaluating non-significant findings.

Purpose of the Study:

• To demonstrate the importance of Bayes factors in interpreting intervention effectiveness.
• To re-analyze the SIPS project data to determine if findings indicate evidence of absence or absence of evidence.

Main Methods:

• Utilized Bayes factors to disambiguate non-significant findings from the SIPS cluster-randomized controlled trial.
• Modeled expected effects and checked the robustness of Bayes factor calculations.

Main Results:

• Individual SIPS trials were uninformative; combined data showed moderate evidence for a null hypothesis (Bayes factor B = 0.24).
• This indicates a lack of effect for the brief intervention compared to control conditions.

Conclusions:

• Bayes factors are essential for scientists to avoid premature judgments on non-significant results.
• Calculating Bayes factors helps determine if data support a null hypothesis or if further data collection is required.