Related Experiment Video
Updated: Jun 12, 2025

E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
Can artificial intelligence read between the lines: Utilizing ChatGPT to evaluate medical students' implicit
Wenqi Geng1, Yinan Jiang1, Wei Zhai2
1Department of Psychological Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
Purpose:
To explore ChatGPT's utility in evaluating medical students' implicit attitudes toward the doctor-patient relationship (DPR).
Materials And Methods:
This study analyzed interview transcripts from 10 medical students, categorizing implicit DPR attitudes into Care and Share dimensions, each with 5 levels. We first assessed ChatGPT's ability to identify DPR-related textual content, then compared grading results from experts, ChatGPT, and participants' self-evaluations. Finally, experts evaluated ChatGPT's performance acceptability.
Results:
ChatGPT annotated fewer DPR-related segments than human experts. In grading, pre-course scores from experts and ChatGPT were comparable but lower than self-assessments. Post-course, expert scores were lower than ChatGPT's and further below self-assessments. ChatGPT achieved an accuracy of 0.84-0.85, precision of 0.81-0.85, recall of 0.84-0.85, and F1 score of 0.82-0.84 for attitude classification, with an average acceptability score of 3.9/5.
Conclusions:
Large language models (LLMs) demonstrated high consistency with human experts in judging implicit attitudes. Future research should optimize LLMs and replicate this framework across diverse contexts with larger samples.
More Related Videos
Related Concept Videos
Surveys
Attitudes
Assessment of the Gastrointestinal System II: Health Perception Pattern
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Stereotype Content Model
Stereotypes, Prejudice, and Discrimination
Barriers to Effective Communication II
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...

