Using receiver operating characteristic analysis to evaluate the accuracy in predicting future quality grade from...

Ultrasound estimates of marbling score were collected on 144 calves that averaged 219 d of age and 219 kg. Those estimates were correlated (r2 = .32, P < .001) with carcass marbling at slaughter 252 d later. Receiver operating characteristic (ROC) analysis provides a method to determine critical operating points for selecting outcome groups with specified percentages of the desired quality grades. When an a priori estimate of the prevalence of Choice or premium choice can be made, the ROC procedures incorporate error rates to estimate the percentage of the herd that will be selected in an outcome group. A unique feature of the ROC output is the ability to conduct a cost/benefit analysis. The ROC analysis indicated that the initial marbling estimates were 78+/-4% accurate in classifying future quality grade and predicting whether an animal would grade USDA Choice. A similar accuracy (76+/-4%) was observed in predicting whether an animal would attain USDA Average Choice or higher (premium choice). A decision matrix was also examined in which sensitivity and specificity in predicting Choice were 90% and 46%, respectively. Relative values of those diagnostic measures were reversed in predicting premium choice (39% and 86%, respectively). Evaluation of the different methods indicated that ROC analysis may have advantages over either traditional regression analysis or contingency tables for evaluating ultrasound procedures.