Asymptotic versus exact methods in the analysis of contingency tables: Evidence-based practical recommendations

Area of Science:

• Statistics
• Statistical Analysis
• Data Science

Background:

• Significance tests in contingency tables face controversy due to discrepancies between asymptotic and exact p-values.
• These discrepancies are often linked to the magnitude of expected frequencies in the data.

Purpose of the Study:

• To investigate the causes of disagreement between asymptotic and exact p-values in contingency table analysis.
• To propose a novel method for exact tests that addresses limitations of current approaches.
• To evaluate the accuracy of asymptotic approximations under revised exact testing frameworks.

Main Methods:

• Analysis of Pearson's X^2 statistic and its asymptotic distributions.
• Development of an exact testing method by integrating nuisance parameters under full-multinomial or product-multinomial models.
• Comparison of exact test results with asymptotic approximations.

Main Results:

• Discrepancies between asymptotic and exact p-values are not solely dependent on expected frequencies but also on the discrete distribution of the X^2 statistic.
• The hypergeometric sampling model used in exact tests can artificially create discrepancies.
• The proposed novel method yields exact distributions accurately approximated by asymptotic distributions.
• A single-stage test for residual significance is recommended over a two-stage approach, preserving Type-I error rates.

Conclusions:

• Concerns about the accuracy of asymptotic p-values in contingency table analysis can be resolved through appropriate exact testing methods.
• The novel integration method for nuisance parameters provides accurate results, validating asymptotic approximations.
• A single-stage testing strategy is superior for maintaining statistical rigor and accuracy.