Diagnostics for assumptions in moderate to large simple clinical trials: do they really help?

Area of Science:

• Biostatistics
• Statistical Inference
• Clinical Trial Design

Background:

• Diagnostic tests for statistical assumptions are common before efficacy analysis.
• Hybrid approaches combine diagnostic tests with primary analyses.
• Potential risks of hybrid approaches in statistical analysis are not fully understood.

Purpose of the Study:

• To illustrate the adverse effects of a hybrid approach using a diagnostic test for equality of variance in a two-sample t-test.
• To provide rationale for classical tests being asymptotically non-parametric and robust.
• To caution analysts about the use of hybrid approaches in complex efficacy analyses.

Main Methods:

• Case study analysis of a two-sample t-test scenario.
• Examination of operating characteristics under assumption violations.
• Theoretical rationale for the robustness of classical tests.

Main Results:

• A hybrid approach using a variance equality diagnostic test adversely affected study operating characteristics.
• Classical tests, potentially modified, can be asymptotically non-parametric and robust.
• Rank tests may be less robust than classical tests when assumptions fail.

Conclusions:

• Analysts should exercise caution with hybrid approaches in efficacy analysis.
• Classical statistical tests may offer greater robustness against assumption failures than previously thought.
• The findings are specific to efficacy analysis and do not critique other analytical methods.