Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study.

Area of Science:

• Medical Education Technology
• Artificial Intelligence in Healthcare
• Clinical Case Simulation

Background:

• Evaluating AI-generated medical cases, particularly from large language models like ChatGPT-4, is essential but under-researched.
• The accuracy and educational value of AI-created clinical scenarios require thorough assessment.

Purpose of the Study:

• To assess the educational utility of ChatGPT-4-generated clinical vignettes in Japanese.
• To determine the applicability of AI-generated medical cases in educational settings.

Main Methods:

• A convergent mixed-methods design using a web-based survey of 71 physicians.
• Evaluation of 18 ChatGPT-4-generated Japanese medical cases using 6 quality and utility metrics.
• Statistical analysis (chi-square, Mann-Whitney U, regression) and thematic analysis of qualitative feedback.

Main Results:

• Physicians rated information quality (0.77) and accuracy (0.68) as satisfactory.
• Average scores for educational usefulness (3.55), clinical match (3.70), and terminology accuracy (3.49) were obtained.
• Significant variability in case quality (P<.001) was noted, with suggestions for improving physical findings, natural language, and terminology accuracy.

Conclusions:

• ChatGPT-4-generated medical cases demonstrate significant potential as educational resources in medicine.
• While adequate in quality and accuracy, these AI cases need enhancement in detail precision and realism.
• ChatGPT-4 serves as a valuable adjunctive educational tool, necessitating expert supervision for effective implementation.