A comparison of predictions made by three simulation models of foot-and-mouth disease

To describe results of a relative validation exercise using the three simulation models of foot-and-mouth disease (FMD) in use by the quadrilateral countries (QUADS; Australia, Canada, New Zealand, and United States of America; USA).

A hypothetical population of farms was constructed and, following the introduction of an FMD-like disease into a single farm, spread of disease was simulated using each of the three FMD simulation models used by the QUADS countries. A series of 11 scenarios was developed to systematically evaluate the key processes of disease transmission and control used by each of the three models. The predicted number of infected units and the size of predicted outbreak areas for each scenario and each model were compared using the Kruskal-Wallis test. Agreement among the three models in terms of geographical areas predicted to become infected were quantified using Fleiss' Kappa statistic.

Although there were statistically significant differences in model outputs in terms of the numbers of units predicted to become infected, the temporal onset of infection throughout the simulation period, and the spatial distribution of infected units, these differences were generally small and would have resulted in the same (or similar) management decisions being adopted in each case.

Agreement among the three models in terms of the numbers of premises predicted to become infected, the temporal onset of infection throughout the simulation period, and the spatial distribution of infected premises provides evidence that each of the model developers are consistent in their approach to simulating the spread of disease throughout a population of susceptible individuals. This consistency implies that the assumptions taken by each development team are appropriate, which in turn serves to increase end-user confidence in model predictions.

Relative validation is one of a number of steps that can be undertaken to increase end-user confidence in predictions made by infectious disease models.