Validation of the FAMACHA© eye colour chart using sensitivity/specificity analysis on two South African sheep farms

A validation study of the FAMACHA(©) system for clinical evaluation of anaemia due to Haemonchus contortus was conducted on two commercial sheep farms in the summer rainfall region of South Africa. In this region, the Haemonchus season lasts from October to April. On Farm 1 the system was tested over a period of five successive years in consecutive sets of young stud Merino replacement rams and ewes examined at intervals of 3-5 weeks over each Haemonchus season, under routine farming conditions. When FAMACHA(©) scores of 3, 4, and 5 and haematocrit values of ≤ 22%, ≤ 19%, and ≤ 15% were separately considered to be anaemic, sensitivity on Farm 1 ranged from a maximum of 83% for a haematocrit cut-off of ≤ 15%, to 40% for a haematocrit cut-off of ≤ 22%. Sensitivity increased to 93% when FAMACHA(©) scores of 2, 3, 4, and 5 were considered anaemic at a cut-off value of ≤ 19%, but the positive predictive value decreased to 0.43, indicating that many non-anaemic animals would be treated. The analysis indicated a high level of classification bias on Farm 1, with the animals consistently being classified one FAMACHA(©) category lower (i.e. less anaemic) than reality. On Farm 2 the test was conducted over two successive years in yearling rams evaluated at weekly to fortnightly intervals during each worm season. Every ram judged to be in FAMACHA(©) category 4 or 5 was bled for haematocrit determination, and it was only dewormed with effective anthelmintics if the haematocrit was 15% or lower. When FAMACHA(©) scores of 3, 4, and 5 and haematocrit values of ≤ 22% and ≤ 19% were separately considered to be anaemic on Farm 2, sensitivity ranged from 64% for a haematocrit cut-off of ≤ 22%, to 80% for a cut-off of ≤ 19%. For identical haematocrit cut-off values and proportions of the sampled flock considered to be diseased as for Farm 1, sensitivity was always higher for Farm 2. On the other hand, further analysis of the data indicated that the magnitude of the error on Farm 1 was very consistent on average over the entire trial period. The results of this study indicate that (i) persons introduced to the system should not only be trained, but also be evaluated for accuracy of application; (ii) the sensitivity of the FAMACHA(©) diagnostic system should ideally be evaluated at shorter intervals to avoid production losses due to failure to detect anaemic animals which may be at risk of death; (iii) that calibration of the FAMACHA(©) scoring is essential per individual evaluator, and (iv) that animals should be examined at weekly intervals during periods of the highest worm challenge.