Maximally selected measures of evidence of disease clusters

It is now common to read reports such as 'city A has a childhood cancer rate 30 per cent higher than the national average'. Because the details about how the data was examined can greatly impact the conclusions, the epidemiologist then wants to know more information, such as 'over what time period' and 'how many other cancer types were examined'. For example, city A might have calculated cancer rates for many age groups within different regions of the city over many time intervals, but reported only the highest cancer rate discovered in a particular group. We will refer to such selective reporting as 'maximally selecting measures of evidence of disease clustering', or less formally as 'fishing for statistical significance'. The objective of this paper is to study the behaviour of maximally selected statistics for measuring the extent of clustering in disease outbreaks. The original data is the time and location of each reported case of the disease. In some cases we are only given aggregates of the original data, such as the number of cases during a time period over a given region. We introduce new and review existing methods for correcting for the effect of making maximal selections in disease cluster detection. We consider three main cases with examples. We demonstrate via simulation and analytical approximations that some types of 'fishing' are simple to correct for while others are not.