Diagnostic alignment to optimize inter-rater reliability among lung transplant pathologists

Poor agreement among lung transplant (LTx) pathologists has been reported in the assessment of rejection. In addition to acute rejection (AR) and lymphocytic bronchiolitis (LB), acute lung injury (ALI) and organizing pneumonia (OP) were recently identified as histopathologic risk factors for chronic lung allograft dysfunction (CLAD). Therefore, maximizing inter-rater reliability (IRR) for identifying these histopathologic risk factors is important to guide individual patient care and to support incorporating them in inclusion criteria for clinical trials in lung transplantation.

Nine pathologists across 8 North American LTx centers were surveyed for practices in the assessment of LTx transbronchial biopsies. We conducted 7 diagnostic alignment sessions with pathologists discussing histomorphologic features of CLAD high-risk histopathology. Then, each pathologist blindly scored 75 digitized slides. Fleiss' kappa, accounting for agreement across numerous observers, was used to determine IRR across all raters for the presence of any high-risk finding and each individual entity.

IRR (95% confidence intervals) and % agreement for any high-risk finding (AR, LB, ALI, and/or OP) and each individual finding is as follows: Any Finding, k = 0.578 (0.487, 0.668), 78.9%; AR, k = 0.582 (0.481, 0.651), 79.1%; LB, k = 0.683 (0.585, 0.764), 83.5%; ALI, k = 0.418 (0.312, 0.494), 70.9%; and OP, k = 0.621 (0.560, 0.714), 81.0%.

After prestudy diagnostic alignment sessions, a multicenter group of LTx pathologists seeking to identify histopathology high-risk for CLAD achieved good IRR.