The impact of subjective image quality evaluation in mammography

According to the guidelines, the mammograms obtained in a screening program must be evaluated to ensure that the quality of the images obtained is above 75% of score 1 (perfect and good) and less than 3% of score 3 (inadequate). This is performed by a person (usually a radiographer), so subjectivity may influence the final evaluation of the images. The aim of this study was to evaluate the impact of subjectivity on breast positioning assessments on resultant screening mammograms.

Five radiographers evaluated a total of 1000 mammograms. One radiographer was an expert in assessing mammography images, while the other four evaluators had varying levels of experience. All images were anonymized, and the ViewDEX software was used for visual grading analysis. The evaluators were divided into two groups, each with two evaluators. Each group evaluated 600 images, with 200 images identical between the two groups. All images had already been evaluated by the expert radiographer. All scores were compared using the Fleiss' and Cohen's kappa coefficient and accuracy score.

The results from Fleiss' kappa showed fair agreement in the mediolateral oblique (MLO) projection in the first group of evaluators whereas the other results showed poor agreement. When comparing the results from Cohen's kappa a maximum value of agreement between the evaluators was moderate 0.433 [95% CI 0.264-0.587] for the craniocaudal (CC) projection and 0.374 [95% CI 0.212-0.538] for the MLO projection.

Based on our results, we can conclude that the agreement between all five raters was poor for both CC (κ = 0.165) and MLO (κ = 0.135) projections, based on the results of Fleiss' kappa statistic. The results show that the influence of subjectivity has a great impact on the evaluation of the quality of mammography images.

Thus, the images are evaluated by a person, which has a high impact on subjectivity in the assessment of positioning in mammography. To achieve a more objective assessment of the images and the resulting agreement between the evaluators, we would propose to change the method of assessment. The images could be evaluated by two persons, and in the event of a discrepancy, the images would be evaluated by a third person. A computer programme could also be developed that would allow a more objective evaluation based on the geometric characteristics of the image (angle and length of the pectoral muscle, symmetry, etc.).