Establishing an appropriate mode of comparison for measuring the performance of marbling score output from video...

A beef carcass instrument grading system that improves accuracy and consistency of marbling score (MS) evaluation would have the potential to advance value-based marketing efforts and reduce disparity in quality grading among USDA graders, shifts, and plants. The objectives of this study were to use output data from the Video Image Analysis-Computer Vision System (VIA-CVS, Research Management Systems Inc., Fort Collins, CO) to develop an appropriate method by which performance of video image analysis MS output could be evaluated for accuracy, precision, and repeatability for purposes of seeking official USDA approval for using an instrument in commerce to augment assessment of quality grade, and to use the developed standards to gain approval for VIA-CVS to assist USDA personnel in assigning official beef carcass MS. An initial MS output algorithm was developed (phase I) for the VIA-CVS before 2 separate preliminary instrument evaluation trials (phases II and III) were conducted. During phases II and III, a 3-member panel of USDA expert graders independently assigned MS to 1,068 and 1,242 stationary carcasses, respectively. Mean expert MS was calculated for each carcass. Additionally, a separate 3-member USDA expert panel developed a consensus MS for each carcass in phase III. In phase II, VIA-CVS stationary triple-placement and triple-trigger instrument repeatability values (n = 262 and 260, respectively), measured as the percentage of total variance explained by carcasses, were 99.9 and 99.8%, respectively. In phases II and III, 95% of carcasses were assigned expert MS for which differences between individual expert MS, and for which the consensus MS in phase III only, was < or = 96 MS units. Two differing approaches to simple regression analysis, as well as a separate method-comparability analysis that accommodates error in both dependent and independent variables, were used to assess accuracy and precision of instrument MS predictions vs. mean expert MS. Method-comparability analysis was more appropriate in assessing the bias and precision of instrument MS predictions. Ether-extractable fat percentages (n = 257; phase II) differed among MS (P < 0.05) but were not suitable to predict or validate assigned MS. The performance and reproducibility of expert MS assignment in future evaluations was considered, and an official USDA performance standard was established, to which an instrument must conform to be approved for official on-line MS assessment. The VIA-CVS subsequently was approved to assign MS to carcasses on-line after completion of a 2006 USDA instrument approval trial conducted according to methods developed during completion of this study.