Algorithm to reduce inter-rater variability in assessment of intermacs patient profiles

Intermacs Patient Profiles (IPP) are subjective assessments of patient acuity of illness assigned at the time of durable left ventricular assist device implantation (dLVAD). Previous studies have demonstrated provider variability with IPP assignment. We developed a novel algorithm to derive a patient's IPP acuity of illness to assist clinicians with IPP assignment.

Twenty standardized clinical vignettes were developed characterizing a patient's status at time of evaluation for dLVAD implantation. Vignettes included the full range of patient acuity of illness represented by IPPs. An algorithm to assign IPPs (1-7) was developed and tested among 15 advanced heart failure specialists invited to assign an IPP across two phases: Phase One (based solely on clinical judgment/experience) and Phase Two (using the algorithm). Phases One and Two were separated by > 3 weeks with vignette numbers changed to reduce risk of rater bias. Iinter-rater variability was tested using the Pitman-Morgan Test. We compared the coefficient of variation (CoV) among assessors for each patient vignette before and after using the standardized algorithm.

Of 20 vignettes, 3 vignettes (14, 16, and 17) demonstrated significant reduction in CoV between Phase 1 & 2. One vignette (5) had a significant increase in CoV. The remaining vignettes demonstrated no significant changes in CoV between Phase 1 & 2.

An algorithm assigning IPPs reproduces IPP assignments determined by heart failure specialists with previous experience assigning IPPs. The developed algorithm could be used to assist clinicians with patient decision-making as well as screening patients for randomized clinical trials.