Persistent inter-observer variability of breast density assessment using BI-RADS® 5th edition guidelines

Due to most states' legislation, mammographic density categorization has potentially far-reaching implications, but remains subjective based on BIRADS® guidelines. We aimed to determine 1) effect of BI-RADS® 5th edition (5th-ed) vs 4th-edition (4th-ed) guidelines on reader agreement regarding density assessment; 2) 5th-ed vs 4th-ed density distribution, and visual vs quantitative assessment agreement; 3) agreement between experienced vs less experienced readers.

In a retrospective review, six breast imaging radiologists (BIR) (23-30 years' experience) visually assessed density of 200 screening mammograms performed September 2012-January 2013 using 5th-ed guidelines. Results were compared to 2016 data of the same readers evaluating the same mammograms using 4th-ed guidelines after a training module. 5th-ed density categorization by seven junior BIR (1-5 years' experience) was compared to eight experienced BIR. Nelson et al.'s kappas (κ_m, κ_w), Fleiss' κ_F, and Cohen's κ were calculated. Quantitative density using Volpara was compared with reader assessments.

Inter-reader weighted agreement using 5th-ed is moderately strong, 0.73 (κ_w, s.e. = 0.01), similar to 4th-ed, 0.71 (κ_w, s.e. = 0.03). Intra-reader Cohen's κ is 0.23-0.34, similar to 4th-ed. Binary not-dense vs dense categorization, using 5th-ed results in higher dense categorization vs 4th-ed (p < 0.001). 5th-ed density distribution results in higher numbers in categories B/C vs 4th-ed (p < 0.001). Distribution for 5th-ed does not differ based on reader experience (p = 0.09). Reader vs quantitative weighted agreement is similar (5th-ed, Cohen's κ = 0.76-0.85; 4th-ed, Cohen's κ = 0.68-0.83).

There is persistent subjectivity of visually assessed mammographic density using 5th-ed guidelines; experience does not correlate with better inter-reader agreement.