Using a Direct Observation Tool (TOC-CEX) to Standardize Transitions of Care by Residents at a Community Hospital

Background: High-quality transitions of care are crucial for patient safety in hospitals, yet few undergraduate curricula include transition-of-care training. In 2012, the Wayne State University Office of Graduate Medical Education (WSUGME) required its residency programs to use the SAIF-IR mnemonic (summary, active issues, if-then contingency planning, follow-up activities, interactive questioning, readback) to ensure accurate and uniform handoffs. Subsequent program evaluations indicated that resident awareness and adoption of the mnemonic at our primary clinical site, Ascension Providence Rochester Hospital (APRH), could be improved. According to our institution's 2016 Clinical Learning Environment Review (CLER), 88% of residents reported following a standardized transition of care handoff, and 53% reported that faculty rarely supervised their handoffs. A 2016 WSUGME internal survey also revealed low rates of awareness (7% to 10%) of the mandated mnemonic. WSUGME then created a direct observation tool, the Transitions of Care-Clinical Evaluation Exercise (TOC-CEX), for faculty to monitor resident skill in using the mnemonic and thus standardize transitions of care as a practice habit at APRH. Methods: Since 2014, WSUGME had relied on 2 methods for training residents in the required handoff mnemonic: (1) introduction to the SAIF-IR mnemonic during the WSUGME orientation for all interns and (2) simulations during an objective simulated handoff evaluation activity for all postgraduate year (PGY) 1s and PGY 2s. In 2017, WSUGME innovated a direct observation tool, the TOC-CEX, for adoption by faculty at APRH to assess resident knowledge of and monitor their skill in using the SAIF-IR mnemonic in 3 primary care programs. The total number of possible participants was 138, and the actual number of individuals in the sample was 95. A majority (86%) of the observations during the study period were of PGY 1 residents, and thus the analysis reflects the ratings of 99% of all interns but only 69% of all possible residents. Results: WSUGME found that faculty use of a direct observation instrument in the clinical learning environment during 2017-2019 increased awareness and adoption of the SAIF-IR mnemonic among residents. Using a z-test of equal proportions on resident responses on an internal WSUGME survey, we found a significant rise in the percentage reporting yes to the question "Does your program have a mechanism for monitoring handoffs?" (χ [3]=23.6, P<0.0001) and in the percentage identifying SAIF-IR in response to the question "Does your program endorse a specific mnemonic for organizing the contents of a verbal handoff?" (χ [3]=45.0, P<0.0001). The increase from 2016 to 2017 is the result of the implementation of the TOC-CEX in the interim (question 1: χ [1]=12.4, P<0.0005; question 2: χ [1]=10.1, P<0.0025). Conclusion: Our research found that use of the TOC-CEX to monitor resident handoffs resulted in improved awareness and adoption of the SAIF-IR mnemonic in the clinical learning environment. Program leadership reported that the practice was both feasible and well accepted by residents, faculty, and the APRH chief medical officer as the TOC-CEX became a customary component of APRH organizational culture and was perceived as central to quality patient care.