Statistical methods for in situ hybridization: identification of autoradiographically labelled cells and structures

In situ hybridization experiments frequently use autoradiography to identify labelled structures. Ideally, labelled cells will be overlain with a dense accumulation of particles, allowing one to discriminate them from unlabelled cells easily. However, if noise is high or the density of labelling is low, it can be difficult to distinguish bona fide labelling 'by eye'. In such situations, labelled cells could be overlooked. This paper evaluates two statistical solutions to this problem: (1) a parametric method proposed by Hashimoto and co-workers and (2) Wang & Wessendorf's non-parametric method using contingency testing (i.e. the chi-square or Fisher's exact tests). The Hashimoto method determines the mean and standard deviation of the density of background labelling, using sense-strand controls as the source of background levels. Cells labelled at densities greater than two standard deviations above the mean (P < 0.0455) are defined as significantly labelled. Contingency testing determines whether the grain density over a cell is significantly higher than that over the remainder of the image. When compared, the two methods gave similar results. The Hashimoto method may be more sensitive if most cells are labelled but contingency testing requires no assumptions about the uniformity of non-specific labelling.