The dual test: safeguarding p-value combination tests for adaptive designs

Area of Science:

• Biostatistics
• Clinical Trial Design
• Statistical Inference

Background:

• Modern adaptive trial designs often use weighted p-values for overall hypothesis testing, offering flexibility.
• However, these combination tests can violate statistical principles, potentially leading to paradoxical conclusions where a positive effect is 'proven' despite a negative average effect.

Purpose of the Study:

• To modify combination tests in adaptive designs to prevent illogical conclusions.
• To introduce a dual test that safeguards against unconvincing results while maintaining design flexibility and type I error control.

Main Methods:

• The proposed dual test requires both a weighted combination test and a naive test (ignoring adaptations) to achieve statistical significance.
• The study analyzes the properties of the dual test, including its conservativeness and power compared to the combination test.
• Specific applications to two-stage sample size reestimation (SSR) are explored, detailing sample size modifications for equal power.

Main Results:

• The dual test ensures that illogical conclusions, such as 'proving' a positive effect when the average effect is negative, are avoided.
• The dual test is, by construction, at least as conservative as the combination test.
• For two-stage SSR, rules based on conditional power show ignorable power loss, whereas a decision analytic approach reveals discrepancies.

Conclusions:

• The dual test offers a robust modification to adaptive trial designs, enhancing the reliability of hypothesis testing.
• It preserves the benefits of flexible adaptive designs while adding a crucial layer of statistical validity.
• The dual test is particularly relevant for adaptive designs like two-stage SSR, with conditional power rules demonstrating minimal power loss.