Revisit of test-then-pool methods and some practical considerations

Test-then-pool is a simple statistical method that borrows historical information to improve efficiency of the drug development process. The original test-then-pool method examines the difference between the historical and current information and then pools the information if there is no significant difference. One drawback of this method is that a nonsignificant difference may not always imply consistency between the historical and current information. As a result, the original test-then-pool method is more likely to incorrectly borrow information from the historical control when the current trial has a small sample size. Statistically, it is more natural to use an equivalence test for examining the consistency. This manuscript develops an equivalence-based test-then-pool method for a continuous endpoint, explains the relationship between the two test-then-pool methods, explores the choice of an equivalence margin through the overlap probability, and proposes an adjustment to the nominal testing level for controlling type I error under the true consistency scenario. Furthermore, the analytical forms of the type I error and power for the two test-then-pool methods are derived, and practical considerations for using them are presented.