Related Experiment Video
Updated: Jun 28, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Performance of ChatGPT vs. HuggingChat on OB-GYN Topics
Gabrielle Kirshteyn1, Roei Golan2, Mark Chaet3
1Obstetrics and Gynecology, Florida State University College of Medicine, Tallahassee, USA.
Large language models like ChatGPT and HuggingChat show promise in obstetrics and gynecology (OB-GYN) medical questions. This study found no significant performance difference between these AI tools on OB-GYN exams.
Area of Science:
- Medical Artificial Intelligence
- Medical Education Technology
Background:
- Large language models (LLMs) offer potential in medicine, but their reliability in obstetrics and gynecology (OB-GYN) is not well understood.
- Evaluating LLM performance is crucial for safe and effective integration into medical practice.
Purpose of the Study:
- To measure and compare the performance of ChatGPT and HuggingChat on OB-GYN medical examination questions.
- To provide insights into the effectiveness of these AI tools in a specialized medical field.
Main Methods:
- ChatGPT and HuggingChat were tested using two standardized multiple-choice question banks: National Board of Medical Examiners (NBME) and Association of Professors of Gynecology & Obstetrics (APGO) uWISE.
- Responses were analyzed for correctness, and performance differences were statistically evaluated using a two-proportion z-test.
Main Results:
- No statistically significant difference in performance was found between ChatGPT and HuggingChat on either examination.
- ChatGPT achieved 90% accuracy on Test 1 (NBME) and 70% on Test 2 (APGO uWISE).
- HuggingChat achieved 85% accuracy on Test 1 and 62% on Test 2.
Conclusions:
- Both ChatGPT and HuggingChat demonstrate considerable promise for medical applications, particularly in OB-GYN.
- Understanding the capabilities and limitations of AI tools is essential for their appropriate use in healthcare and medical education.
More Related Videos
06:28E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
14:43A Novel Method for Involving Women of Color at High Risk for Preterm Birth in Research Priority Setting
Published on: January 12, 2018
Related Concept Videos
Gonadal and Placental Hormones
In males, testosterone is the primary gonadal androgen. It plays a central role in the maturation of male reproductive organs — the penis and testes. Additionally, testosterone is instrumental in the development of secondary sexual characteristics — a deep voice as well as facial and pubic hair...
Hormonal Regulation
Hormones of the Pituitary Gland
The most abundantly secreted hormone from the anterior lobe is the growth hormone, which controls overall growth by...
Health Information Technology and Healthcare Information System
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Improving Translational Accuracy
Patient-centered Care