Efficiency in sequential testing: Comparing the sequential probability ratio test and the sequential Bayes factor...

Area of Science:

• Psychological research methodology
• Statistical inference
• Sequential hypothesis testing

Background:

• Sequential hypothesis testing allows for early termination of data collection once sufficient evidence is gathered.
• Two prominent sequential methods, the Sequential Probability Ratio Test (SPRT) and the Sequential Bayes Factor Test (SBFT), are increasingly used in psychological research.

Purpose of the Study:

• To compare the Sequential Probability Ratio Test (SPRT) and the Sequential Bayes Factor Test (SBFT) in the context of psychological research.
• To elucidate the similarities and differences between SPRT and SBFT, particularly their underlying mechanisms for evidence monitoring and error control.
• To provide guidance on selecting appropriate sequential designs by balancing efficiency, robustness, and uncertainty quantification.

Main Methods:

• Comparative analysis of SPRT and SBFT, highlighting their shared mathematical framework and philosophical underpinnings.
• Examination of evidence monitoring and error control mechanisms within both sequential testing procedures.
• Simulation studies to assess the efficiency of SPRT and SBFT under varying statistical model specifications and population truths.

Main Results:

• SPRT and SBFT, despite different origins, can be viewed as instances of a unified sequential hypothesis testing framework.
• Both methods employ similar strategies for monitoring accumulating evidence and controlling decision errors.
• The relative efficiency of SPRT and SBFT is contingent upon the precise statistical models used and the true state of the population.

Conclusions:

• Researchers designing sequential studies must carefully consider the trade-offs between test efficiency, robustness to model misspecification, and accurate quantification of uncertainty.
• Guidance is offered for making informed design decisions in sequential hypothesis testing, tailored to specific research needs and preferences.
• The unified framework provides a valuable perspective for understanding and applying diverse sequential testing procedures in psychological science.