Related Experiment Video
Updated: Dec 24, 2025
The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living
Published on: July 28, 2022
Fast, Easy, and Good: Assessing Entrustable Professional Activities in Psychiatry Residents With a Mobile App
John Q Young1, Matthew McClure2
1J.Q. Young is professor, Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York.
Problem:
Entrustable professional activities (EPAs) can be used to operationalize competency-based medical education. Mobile apps can efficiently capture feedback based on direct observation. To leverage the benefits of both, the authors developed an assessment tool that combines EPAs with mobile technology.
Approach:
The authors designed an app to collect EPA data based on direct observation using human-technology interface guidelines. Data collected in the app included: name of resident, the 13 end-of-training EPAs for psychiatry, entrustment ratings, and corrective narrative feedback. The app was implemented in an outpatient continuity clinic for second-year psychiatry residents over a 10-month period between September 2017 and June 2018. Ten faculty-resident dyads piloted the app. To assess the feasibility, utility, and validity of this intervention, the authors examined 3 outcomes: (1) utilization (mean time to complete each assessment; percentage of dyads who completed 10 assessments), (2) quality of the comments (proportion of comments that were behaviorally specific and actionable), and (3) correlation between entrustment level and resident experience (defined as days elapsed since the beginning of the experience).
Outcomes:
A total of 99 assessments were completed during the pilot. Mean time to complete an assessment was 76 seconds (standard deviation = 50 seconds, median = 67 seconds). Only 6 of the 10 dyads completed at least 10 assessments. Of all comments, 95% (94) were behaviorally specific and actionable and 91% (90) were corrective. Entrustment scores correlated moderately with resident experience (r = 0.43, P < .001).
Next Steps:
The authors' EPA mobile app was efficient, generated high-quality feedback, and produced entrustment scores that improved as the residents gained experience. Challenges included uneven adoption. Looking forward, the authors plan to examine the enablers and barriers to adoption from an implementation science perspective.
