Related Experiment Video
Updated: Jun 29, 2025

E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
ChatGPT's Response Consistency: A Study on Repeated Queries of Medical Examination Questions
Paul F Funk1, Cosima C Hoch2, Samuel Knoedler3
1Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Jena, Friedrich Schiller University Jena, Am Klinikum 1, 07747 Jena, Germany.
ChatGPT 4 demonstrates significantly higher accuracy and consistency in answering medical questions compared to ChatGPT 3.5. This highlights the potential of advanced AI in medical education and decision-making support.
Area of Science:
- Medical Education
- Artificial Intelligence in Medicine
- Clinical Decision Support
Background:
- Artificial intelligence (AI) tools like ChatGPT are increasingly used in medical education and research.
- Ensuring the reliability of AI-generated information is crucial in the medical field.
Purpose of the Study:
- To evaluate the accuracy and consistency of ChatGPT 3.5 and ChatGPT 4 in answering medical examination questions.
- To compare the performance of different versions of ChatGPT for medical applications.
Main Methods:
- 450 medical examination questions were input into ChatGPT 3.5 and ChatGPT 4 three times each.
- Responses were collected and statistically analyzed for accuracy and consistency.
Main Results:
- ChatGPT 4 achieved 85.7% accuracy, significantly outperforming ChatGPT 3.5's 57.7% (p < 0.001).
- ChatGPT 4 showed higher consistency, answering 77.8% correctly across rounds, compared to 44.9% for ChatGPT 3.5 (p < 0.001).
Conclusions:
- ChatGPT 4 offers improved accuracy and dependability for medical education and potential clinical decision-making.
- Human healthcare remains essential, and continuous assessment is vital for integrating AI in medicine.
Related Concept Videos
Data Collection III
The principles to begin the physical assessment include conducting a comprehensive or problem-related history in a quiet, well-lit room, emphasizing privacy and comfort for the...
Assessment of the Gastrointestinal System II: Health Perception Pattern
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Assessment of the Cardiovascular System I: Subjective Data
Initial Enquiry
Ask the patient about their primary concern and thoroughly explore all reported symptoms.
Medical History
Investigate past illnesses affecting the cardiovascular system, such as angina, anemia, rheumatic fever, congenital heart disease, stroke, thrombophlebitis, dysrhythmias, varicosities
Inquire about symptoms...
Assessment of the Gastrointestinal System I: Subjective Data
Health History
The initial step in assessing the GI system is obtaining a comprehensive health history. This includes inquiring about the patient's history or presence of problems...
Longitudinal Research
Clinical Trials
There are four phases in a clinical trial. A phase one...

