Finite-Sample Two-Group Composite Hypothesis Testing via Machine Learning

Area of Science:

• Statistics
• Machine Learning
• Clinical Trial Design

Background:

• Composite hypothesis testing often lacks uniformly most powerful (UMP) unbiased tests, especially with finite sample sizes.
• Adaptive clinical trials are crucial for situations with limited prior information, like COVID-19 research, but enhancing testing power remains a challenge.

Purpose of the Study:

• To develop an automatic framework for constructing powerful hypothesis testing statistics and critical values using machine learning.
• To enhance statistical power in finite-sample hypothesis testing, particularly for composite hypotheses and adaptive trial designs.

Main Methods:

• Proposed an automatic framework utilizing machine learning methods to construct test statistics and critical values.
• Illustrated the framework's performance using Deep Neural Networks (DNN).
• Evaluated the method through simulations and case studies of adaptive designs.

Main Results:

• The proposed machine learning framework automatically constructs test statistics and critical values.
• Demonstrated satisfactory power in finite-sample settings using DNN.
• The method proved to be general, automatic, and prespecified.

Conclusions:

• Machine learning offers an effective approach to enhance power in finite-sample composite hypothesis testing.
• The developed framework provides a practical solution for complex statistical problems, including adaptive clinical trials.
• The method is generalizable and demonstrates superior performance in enhancing statistical power.