Overview The Fisher exact test of significance may be used in place of the chi-square test in 2-by-2 tables, particularly for small samples. It tests the probability of getting a table as strong as the observed or stronger simply due to the chance of sampling, where "strong" is defined by the proportion of cases on the diagonal with the most cases. Though usually employed as a one-tailed test, it may be computed as a two-tailed test as well. The Fisher exact test is sometimes known as the Fisher-Irwin test as it was developed at the same time by Fisher, Irwin, and Yates in the 1930's. SPSS automatically computes Fisher's exact test in addition to chi-square for 2x2 tables when the table has a cell with an expected frequency of less than 5. Note that the extension of the Fisher exact test to r-by-c tables proposed by Fisher is also called the Fisher-Freeman-Halton test. For Fisher's exact test for r-by-c tables, SPSS requires installation of the Exact Tests add-on module. This, however, is rarely used. Rather, r-by-c tables are normally tested for significance using the chi-square test, possibly incorporating Yates' correction. Applied to the same 2-by-2 small sample data, uncorrected chi-square tends to underestimate the probability of observed cell counts, increasing Type I errors (false positive findings). Corrected chi-square will normally lead to the same significance decision as the Fisher exact test (see Sheskin, 2007: 637).

Contents Key concepts and terms Assumptions Frequently asked questions Bibliography







Key Concepts and Terms

• Notation. For notation in the steps outlined below, let a, b, c, and d be cell entries; let r and c be row and column totals; and let n be sample size, as in the table below:

  a	b	r1
  c	d	r2
  c1	c2	n

• Computation of Fisher's exact test
  Fisher's exact test directly computes p, the probability of getting a table as strong as the observed table or stronger. This requires computing Fisher's formula below for the given table and all stronger tables, then summing the separate p's to get the total probability of a table that strong or stronger, as explained below.
  1. Specify the observed table and all stronger tables. Reduce the cell with the lowest count by 1 in steps to create the tables stronger than the observed table:

     Observed Table	Next Stronger	Strongest Table
     7 2 9 5 6 11 12 8 20	8 1 9 4 7 11 12 8 20	9 0 9 3 8 11 12 8 20

  2. Compute the probability by Fisher's exact test:
     p = (r₁!r₂!c₁!c₂!)/n!a!b!c!d!

     p_observed = 9!11!12!8!/20!7!2!5!6! = .132

     p_stronger = 9!11!12!8!/20!8!1!4!7! = .024

     p_strongest = 9!11!12!8!/20!9!0!3!8! = .001

     p_total (one-tailed) = .157

     (Recall factorial, denoted by !, is the arithmetic operation of multiplying 1 times 2 times 3, etc., up to n, for the case of n!)

  3. Interpreting Fisher's p is straightforward. In the example above, p is .157, meaning that there is a 15.7% chance that, given the sample size and distribution of the observed table, that we would get a table as strong or stronger by chance of sampling alone. Since social scientists ordinarily consider .05 to be the cutoff for acceptability of significance levels, we would conclude that the distribution in the observed table cannot be said to be significantly different from chance.

  4. SPSS output. SPSS output is selected from the menu system by Analyze, Descriptive Statistics, Crosstabs, then under the Statistics button, check chi-square to obtain the output below:
     




Assumptions

• Random sampling is assumed, as in all significance tests.

• A directional hypothesis is assumed in the common one-tailed version of the test illustrated above, though SPSS also generates a two-tailed non-directional test statistic as well. In the example above, a one-tailed test is employed which considers distributions as strong or stronger than that observed. Note that the default in Fisher exact tests is one-tailed and the default in chi-square tests is two-tailed, a difference which may cause confusion or even misrepresentation of significance unless the researcher is careful to state which type of test is in use.

• Independent observations. It is assumed that the value of the first person or unit sampled has no effect on the value for the second person or unit. Pooling data from before-after tests or matched samples would violate this assumption, for instance.

• Mutual exclusivity. A given case may fall in only one cell in the table.

• Dichotomous level of measurement. It is assumed that the 2-by-2 table is formed by true dichotomies and does not, for instance, result from missing rows and columns in a larger table or from binning an interval variable.

• Fixed marginals. The rationale for the Fisher exact test assumes that the marginals are fixed by the researcher in advance, as in an experiment. This assumption is widely ignored in practice.




Frequently Asked Questions

• Where is the Fisher exact test found in SPSS?
  It is located under Analyze, Descriptive Statistics, Crosstabs; click the Statistics button in the Crosstabs dialog box; check Chi-square. This causes SPSS to computer Pearson's chi-square, the likelihood-ratio chi-square, Yates' corrected chi-square, and if a cell has an expected frequency < 5 and the 2-by-2 table does not result from missing rows or columns in a larger table, Fisher's Exact Test will be computed.

• What is Tocher's modification of the Fisher exact test?
  It is clear that all stronger tables than the observed contribute to the probability in the .05 tail of the one-tailed test which is the Fisher exact test. However, Tocher raises an issue about whether the probability of the observed table should always be included. He argues that there is a random sampling chance factor involved in any particular observed distribution. This leads to the conclusion that there is a chance factor involved in whether the probability associated with the observed table (p=.132 in the example above) should be included in the calculation of the total p used in the exact test.

  In the example above, the next stronger and strongest tables have a combined probability of .025. The observed table has a probability of .132. When we add in this observed probability we get .157, which, being more than the .05 significance cutoff, leads us to conclude the relationship in the table is not significant. However, Tocher wishes to call attention to the fact that part or the .132 probability is below the .05 level (.05 - .025 = .025 is below), and part is above (.132 - .05 = .082 is above). Is the relationship in our particular sample assumed to be in the part above or below the cutoff level? Tocher's modification of the Fisher exact test is to use a random number process proportionate to the parts above and below the cutoff to determine whether or not to add the probability of p_observed as part of p_total. If this process leads us to add it, as Fisher's exact test already does, then we conclude the relationship is not significant. However, if the process leads us not to add it, then the resulting p_total is only .025, well below the .05 cutoff, and we consider that the relationship is significant.

  How does Tocher's random number process work? In this example, the proportion of the probability of the observed table that is within (under) the .05 cutoff is .025/.132 = .189. Consulting a pseudo random number generator, as in SPSS, we get a random number in the range 0 to 1. If it is .189 or less, we do not add the observed probability to the total, otherwise we do.

• Is the two-tailed test statistic for Fisher's Exact Test simply double the one-tailed value?
  No. Fisher's exact test does not have a symmetrical sampling distribution. For instance, in the example above, the two-tailed probability level is .197, compared with the one-tailed value of .157. The two-tailed p value is the sum of all p's of all tables whose p's are less than or equal to the p of the observed table. As the two-tailed level is always greater than the one-tailed level, a table which is significant by one-tailed Fisher's exact test is not necessarily significant by a two-tailed Fisher's exact test.




Bibliography

• Sheskin, David J. (2007). Handbook of parametric and nonparametric statistical procedures, 4th ed.. Boca Raton, FL: Chapman & Hall. CRC.
• Siegel, Sidney (1956). Nonparametric statistics for the behavioral sciences. NY: McGraw-Hill. A standard reference work.

Copyright 1998, 2008 by G. David Garson.
Last updated 4/23/08.