Designing a Video-based Remote Clinical Consultation Simulation With Artificial Intelligence-based Communication...

The aim of this research is to gather and analyze design ideas for a video-based remote clinical consultation application that uses AI to detect communication elements, such as nonverbal cues, during simulated consultations. The application is intended for use by health professions students and professionals in simulation-based training to develop nonverbal communication and video consultation skills. It provides feedback based on predefined rules applied to communication detected by AI. We conducted semi-structured interviews with 11 health care professionals and educators to explore their perspectives on video consultations, telehealth training in health professions education, and the methods used to teach nonverbal communication in these programs. The interview data were analyzed using inductive thematic analysis. The findings present key themes that can be used to inform design decisions for simulated video consultation situations, automated recognition of nonverbal communication in video consultation simulation, and feedback derived from it. We found that more objective measures of communication skills could enhance health professions education simulations. In addition, automatic feedback should focus on highlighting positive aspects of communication, limit low-level detail, and provide contextual information to support reflection. Finally, incorporating emotionally charged scenarios is valuable for teaching nonverbal communication skills.