Usability and Utility of Virtual and In-Person Direct Observation of Resident-Patient Encounters

Area of Science:

• Medical Education
• Clinical Skills Assessment
• Technology in Healthcare

Background:

• Direct observation (DO) of clinical encounters is crucial for medical education.
• Barriers to in-person observation of encounters (IOE) are known.
• Virtual observation of encounters (VOE) is an emerging alternative with limited research.

Purpose of the Study:

• To compare virtual observation of encounters (VOE) with in-person observation of encounters (IOE).
• To evaluate usability, utility, and acceptability of VOE versus IOE for pediatric residents.
• To assess perspectives of residents, observers, and families.

Main Methods:

• Randomized cross-over study comparing VOE and IOE in hospital medicine.
• Surveys based on Technology Acceptance Model, System Usability Scale, and Mini Clinical Evaluation Exercise were used.
• Data collected from residents, observers, and families; analyzed using Wilcoxon signed-rank test.

Main Results:

• No significant differences in time, effort, comfort, or perceived performance between VOE and IOE.
• IOE received higher ratings for observation adequacy (interview, examination, rapport, efficiency, competence).
• Observers with technology issues preferred IOE; those without preferred VOE. Families were comfortable with both formats.

Conclusions:

• Both IOE and VOE demonstrate high usability and utility across all participant groups.
• IOE is generally preferred for its observational adequacy and overall preference.
• VOE is a practical alternative to IOE when in-person observation is challenging, provided technology is reliable.