Spatial properties of retinal mosaics: an empirical evaluation of some existing measures

Mosaics of neurons are usually quantified by methods based on nearest-neighbor distance (NND). The commonest indicator of regularity has been the ratio of the mean NND to the standard deviation, here termed the 'conformity ratio.' However, an accurate baseline value of this ratio for bounded random samples has never been determined; nor was its sampling distribution known, making it impossible to test its significance. Instead, significance was assessed from goodness-of-fit to a Rayleigh distribution, or from another ratio, that of the observed mean NND to an expected mean predicted by theory, termed the dispersion index. Neither approach allows for boundary effects that are common in experimental mosaics. Equally common are 'missing' neurons, whose effects on the statistics have not been studied. To address these deficiencies, random patterns and real neuronal mosaics were analyzed statistically. Ns independent random-point samples of size Np were generated for 13 Np values between 25 and 6400, where Ns x Np > or = 144,000. Samples were generated with rectangular boundaries of aspect ratio 1:1, 1:5, and 1:10 to examine the influence of sample geometry. NND distributions, conformity ratios, and dispersion indices were computed for the resulting 45,997 independent random patterns. From these, empirical sampling distributions and critical values were determined. NND distributions for small-to-medium, bounded, random populations were shown to differ significantly from Rayleigh distributions. Thus, goodness-of-fit tests are invalid for most experimental mosaics. Charts are presented from which the significance of conformity ratios or dispersion indices can be read directly. The conformity ratio reacts conservatively to extremes of sample geometry, and provides a useful and safe test. The dispersion index is nonconservative, making its use problematic. Tests based on the theoretical distribution of the dispersion index are unreliable for all but the largest samples. Random deletions were also made from 33 real retinal ganglion cell mosaics. The mean NND, conformity ratio, and dispersion index were determined for each original mosaic and 36 independent samples at each of nine sampling levels, retaining between 90% and 10% of the original population. An exclusion radius, based on a spatial autocorrelogram, was also calculated for each of these 10,725 mosaic samples. The mean NND was moderately insensitive to undersampling, rising smoothly. The exclusion radius was remarkably insensitive. The conformity ratio and dispersion index fell steeply, sometimes failing to reach significance while half of the cells still remained. For the same 33 original mosaics, linear regression showed the exclusion radius to be 62 +/- 3% of the mean NND.