Analysing 2 × 2 contingency tables: which test is best?

A survey of five journals of physiology or pharmacology for 2011 showed that Fisher's exact test was used three times as frequently as Pearson's Chi-squared test. I shall argue that neither test is appropriate for analysing 2 × 2 tables of frequency in biomedical research. Pearson's test requires that random samples are taken from defined populations. The resultant 2 × 2 table is described as unconditional because neither the row nor column marginal totals are fixed in advance. Fisher's test requires the rare condition that both row and column marginal totals are fixed in advance. The resultant 2 × 2 table is described as doubly conditioned. However, the most common design of biomedical studies is that a sample of convenience is taken and divided randomly into two groups of predetermined size. The groups are then exposed to different sets of conditions. The binomial outcome is not fixed in advance, but depends on the result of the study. Thus, only the column (group) marginal totals are fixed in advance and the table is described as singly conditioned. Singly conditioned 2 × 2 tables are best analysed by tests of null hypotheses on the odds ratio (OR = 1) or by tests on proportions (p), such as the relative risk (RR = p(2)/p(1) = 1) or the difference between proportions (p(2) - p(1) = 0). One enormous advantage of these procedures is that they test specific hypotheses. They should be executed in an exact fashion by permutation.