Eye of the beholder: Risk calculators and barriers to adoption in surgical trainees

Accurate risk assessment before surgery is complex and hampered by behavioral factors. Underutilized risk-based decision-support tools may counteract these barriers. The purpose of this study was to identify perceptions of and barriers to the use of surgical risk-assessment tools and assess the importance of data framing as a barrier to adoption in surgical trainees.

We distributed a survey and risk assessment activity to surgical trainees at four training institutions. The primary outcomes of this study were descriptive risk assessment practices currently performed by residents, identifiable influences and obstacles to adoption, and the variability of preference sets when comparing modified System Usability Scores of a current risk calculator to a purpose-built calculator revision. Risk calculator comparison responses were compared with simple and multivariable regression to identify predictors for preferentiality.

We collected responses from 124 surgical residents (39% response rate). Participants endorsed familiarity with direct verbal communication (100%), sketch diagrams (87%), and brochures (59%). The most contemporary risk communication frameworks, such as best-worst case scenario framing (38%), case-specific risk calculators (43%), and all-procedure calculators (52%) were the least familiar. Usage favored traditional models of communication with only 26% of residents regularly using a strategy other than direct verbal discussion or anatomic sketch diagrams. Barriers limiting routine use included lack of electronic and clinical workflow integration. The mean modified System Usability Scores domain scores were widely dispersed for all domains, and no domain demonstrated one calculator's superiority over another.

Risk assessment tools are underutilized by trainees. Of importance, preference sets of clinicians appear to be unpredictable and may benefit more from a customizable, bespoke approach.