Conservative hypothesis tests and confidence intervals using importance sampling.

Area of Science:

• Statistics
• Computational Statistics

Background:

• Importance sampling is widely used for Monte Carlo approximations, including p-value calculations.
• Standard importance sampling p-values may not always yield valid hypothesis tests with controlled Type I error rates.

Purpose of the Study:

• To introduce a simple correction for importance sampling p-values to ensure their validity.
• To demonstrate how corrected p-values can be used for accurate Monte Carlo confidence intervals.

Main Methods:

• A correction method is proposed for importance sampling p-values using the importance weight of the original observation.
• The corrected p-values are shown to provide valid hypothesis tests with Type I error rates at most α.
• Inverting the corrected p-values is used to construct Monte Carlo confidence intervals.

Main Results:

• The corrected importance sampling p-values guarantee valid hypothesis testing.
• The correction provides valuable diagnostic information under the null hypothesis.
• Corrected p-values are crucial for multiple testing and in scenarios where approximation accuracy is hard to evaluate.
• Inverting corrected p-values yields Monte Carlo confidence intervals with nominal significance levels using a single sample.

Conclusions:

• A simple correction ensures the validity of importance sampling p-values for hypothesis testing.
• The corrected p-values offer a robust method for constructing accurate Monte Carlo confidence intervals.
• This approach enhances the reliability of statistical inference in Monte Carlo simulations.