Probability of recurrence of extreme data: an aid to decision-making

Those, such as public health specialists, who have to assess routinely collected data often find it difficult to decide whether or not extreme results are a matter of chance. I describe here a simple method of measuring one's "degree of surprise" for noticeable patterns in ranked data, classified by time and place, to help detect significant departures from random variation. A table of exact probability values is the key to this approach. Designed primarily for reviewing post-hoc data, the method is relevant to published "league tables" of performance indicators. The table has to be used cautiously and the method is intended as a preliminary screen where data are limited or as a supplement to traditional analyses where there is more. Areas of application include public health, audit, and health services management and performance monitoring.