ChatGPT-4o with faculty guidance outperforms AI-only and traditional learning in ultrasonography training: a...

Area of Science:

• Medical Education
• Artificial Intelligence in Medicine
• Diagnostic Imaging

Background:

• Ultrasonography training presents challenges due to its operator-dependent nature and the complexity of image interpretation.
• Multimodal large language models (LLMs) like ChatGPT-4o offer efficient knowledge retrieval but have limitations in analyzing static ultrasonography images.
• Developing effective training strategies is crucial for resident proficiency in ultrasonography.

Purpose of the Study:

• To evaluate the effectiveness of different training modalities for ultrasonography residents.
• To compare traditional learning, AI-only training (ChatGPT-4o), and a blended approach (AI plus faculty guidance).
• To assess the impact of AI integration on resident performance in ultrasonography.

Main Methods:

• A prospective, single-center randomized controlled trial involving 45 first-year ultrasonography residents.
• Residents were allocated to control (traditional resources), AI-only (ChatGPT-4o), or blended (ChatGPT-4o + faculty tutorials) groups.
• A 3-week intervention was followed by a 150-item examination assessing pure-text and image-based knowledge.

Main Results:

• The blended group achieved significantly higher overall scores (128.40 ± 18.25) compared to AI-only (119.87 ± 19.11) and control (110.60 ± 20.45) groups (P=0.02).
• Superior performance was observed in pure-text questions (P=0.03) and specific areas like obstetrics/gynaecology (P=0.04) and superficial organ ultrasonography (P=0.047) in the blended group.
• ChatGPT-4o demonstrated 85% accuracy on text-based questions but only 47% on image-based questions, highlighting AI's limitations in visual interpretation.

Conclusions:

• A faculty-guided, AI-integrated strategy enhances short-term ultrasonography resident performance compared to AI-only or traditional methods.
• The study suggests that while AI can supplement learning, its effectiveness in static ultrasound image interpretation remains limited.
• Blended learning approaches integrating AI with expert faculty guidance appear most beneficial for ultrasonography education.