Use of the Consultation Letter Rating Scale among Geriatric Medicine Postgraduate Trainees

The implementation of competency-based evaluations increases the emphasis on in-training assessment. The Consultation Letter Rating Scale (CLRS), published by the Royal College of Physicians and Surgeons of Canada, is a tool that assesses written-communication competencies. This multisite project evaluated the tool's validity, reliability, feasibility, and acceptability for use in postgraduate geriatric medicine training.

Geriatric medicine trainees provided consultation letters from the 2017-2018 academic year. Geriatricians reviewed a standardized module and completed the tool for all the deidentified letters. The reviewers recorded the time used to complete the tool for each letter and completed a survey on content validity. Trainees completed a survey on the tool's usefulness. Responses were reviewed independently by two authors for thematic content. The unweighted and the weighted κ were used to measure interrater reliability.

A total of 10 of 11 (91%) eligible trainees each provided five letters that were reviewed independently by six geriatricians, leading to a total of 300 assessments. A very small portion (4% [N = 12]) of assessments was incomplete. An average of 4.82 minutes (standard deviation = 3.17) was used to complete the tool. There was high interrater agreement for overall scores, with a multiple-rater weighted κ of 83% (95% confidence interval = 76%-89%). The interrater agreement was lower for the individual components. Both raters and trainees found the comments more useful than the numerical ratings.

Our results support the use of the CLRS for facilitating feedback on the quality of consult letters to improve written-communication competencies among geriatric medicine trainees. J Am Geriatr Soc 67:2157-2160, 2019.