Related Experiment Video
Updated: Jun 2, 2026
Evaluation of Hepatic Glucose Production in a Polycystic Ovary Syndrome Mouse Model
Published on: March 5, 2022
Guideline Adherence in the Diagnosis of Polycystic Ovary Syndrome Among Adolescents: Comparing Obstetricians and
Nisan Helin Donmez1, Hanne Bulat Cim1, Esmehan Aysit2
1Department of Obstetrics and Gynecology, Prof. Dr. Süleyman Yalçın City Hospital, Istanbul, Türkiye.
Study Objective:
Polycystic ovary syndrome (PCOS) in adolescence is a condition associated with long-term cardiovascular, metabolic, and psychological outcomes. The diagnosis of PCOS in adolescence poses diagnostic challenges for obstetricians and gynecologists due to pubertal variability and the use of criteria distinct from adult standards. Inter-clinician variability and the role of large language models (LLMs) in this context remain insufficiently explored. This study evaluated guideline adherence among physicians and LLMs using a vignette-based design.
Methods:
Ten guideline-based adolescent PCOS vignettes (ESHRE/ASRM 2023), validated by three obstetrics and gynecology specialists, were administered to obstetricians and gynecologists in Türkiye (n = 150). The same vignettes were presented to four LLMs (ChatGPT-5.2, Gemini 3 Flash, Grok 4, DeepSeek V3.2) via a standardized prompt using two independent user accounts in standard and reasoning modes. Inter-account consistency was assessed using accounts.
Results:
Physicians achieved a mean of 5.0 ± 1.9 correct responses out of 10 vignettes (each scored as 1 point), whereas LLMs demonstrated significantly higher performance (7.6 ± 1.1; P < .001. Physicians with national assisted reproductive technology certification achieved higher scores in difficult cases. Inter-rater agreement among physicians was low (κ = 0.127, κ = 0.032, κ = 0.027). ChatGPT-5.2 (reasoning mode) and Gemini 3 Flash (standard mode) showed high inter-account agreement (P < .001). Reasoning mode increased overall LLM accuracy.
Conclusion:
Significant inter-clinician guideline discordance was observed in adolescent PCOS diagnosis. Strengthening this domain is critical for adolescent health. Given their nondeterministic and evolving architectures, LLMs should be positioned as adjunctive tools supporting clinical judgment rather than as independent guideline authorities.
