Artificial Intelligence in Diagnosing and Managing Vascular Surgery Patients: An Experimental Study Using the GPT-4
Vangelis G Alexiou1, Bauer E Sumpio2, Areti Vassiliou3
1Department of Surgery - Vascular Surgery Unit, University Hospital of Ioannina, Ioannina, Greece; Alfa Institute of Biomedical Sciences (AIBS), Athens, Greece.
Annals of Vascular Surgery
|November 25, 2024
Summary
Artificial intelligence (AI) chatbots show potential in vascular surgery diagnosis and management, answering over 65% of clinical questions correctly. However, careful analysis of AI reasoning is crucial for clinical validity.
Area of Science:
- Artificial Intelligence in Medicine
- Clinical Decision Support Systems
- Vascular Surgery
Background:
- Artificial intelligence (AI) and machine learning are transforming scientific fields.
- Natural language processing (NLP) enables AI to understand and respond to complex inquiries.
- Investigating AI chatbot application in vascular surgery patient diagnosis and management.
Purpose of the Study:
- Evaluate the performance of the GPT-4 AI model in vascular surgery scenarios.
- Assess AI's accuracy in diagnosing conditions and selecting treatments.
- Identify limitations and areas for improvement in AI clinical applications.
Main Methods:
- Experimental study using GPT-4 AI model.
- 57 clinical scenarios from a vascular surgery textbook were used.
- AI was prompted to identify symptoms, diagnose, and suggest therapies; answers were scored.
Main Results:
- GPT-4 answered over 65% of 385 questions correctly.
- No statistically significant performance differences across 13 vascular surgery topics.
- Errors included misinterpreting complex information (27%), outdated data (14%), and context/nuance issues (11%).
Conclusions:
- GPT-4 shows potential for clinically relevant answers in vascular surgery.
- AI reasoning requires careful analysis for accuracy and clinical validity.
- AI language models are valuable supportive tools, not standalone solutions.
