Related Experiment Video
Updated: Feb 17, 2026
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Which bot is best? Accuracy and readability of generative artificial intelligence assistant responses on adrenal
Natalie A Moreno1, Austin Dixon2, Sophie Y Dream3
1Department of Surgery, Division of Endocrine Surgery, Johns Hopkins University School of Medicine, Baltimore, MD, USA.
Introduction:
Patients increasingly use generative artificial intelligence assistants (chatbots) for medical information. We examined surgeon-perceived accuracy, reliability, and readability of chatbot responses to adrenal nodule queries.
Methods:
Six commonly asked adrenal nodule questions were input into five chatbots. Blinded answers were reviewed by 10 endocrine surgeons for correctness and reliability (6-point Likert scale) and content structure (3-point Likert scale). One-way ANOVAs and Tukey adjusted post hoc analyses tested differences. Inter-rater reliability across surgeon ratings was assessed by two-way intraclass correlation coefficient. Reading grade levels were assessed using Lexile Text Analyzer (MetaMetrics, Inc).
Results:
Correctness, reliability, and jargon use differed significantly (p ≤ 0.05). Perplexity scored highest for correctness, reliability, and thoroughness but used the most jargon. Gemini scored lowest on all domains, but used the least jargon. Mean Lexile reading level was 8th-10th grade.
Conclusion:
Chatbot responses about adrenal nodules vary significantly. More accurate, reliable, and thorough answers contained excess jargon, but all responses exceeded recommended patient education levels.
Related Concept Videos
01:27Adrenal Gland Disorders
Adrenal insufficiency, characterized by insufficient cortisol and aldosterone production, leads to conditions like Addison's disease. This disorder, affecting the adrenal cortex, exhibits symptoms such as skin bronzing, dehydration, low blood pressure, fatigue, and weight loss. Congenital adrenal hyperplasia, a genetic ailment causing...
01:17Anatomy of the Adrenal Glands
These glands possess a distinctive yellow tinge due to the stored cholesterol and fatty acids required for hormone synthesis. They are encased in a fibrous capsule and cushioned by fat.
The adrenal gland comprises two distinct...
01:31Hormones of the Adrenal Glands
The adrenal cortex, a powerhouse of hormone synthesis, generates over two dozen corticosteroid hormones. The zona glomerulosa produces mineralocorticoids, exemplified by aldosterone, influencing the electrolyte composition of body fluids. The synthesis of glucocorticoids such as cortisol and...
01:27Sympathetic Pathways: Collateral Ganglia and Adrenal Medulla
Collateral Ganglia
Sympathetic preganglionic axons reach the collateral ganglia along the route of splanchnic nerves. These nerves bypass the sympathetic trunk and communicate with sympathetic postganglionic neurons housed in the prevertebral ganglia. These ganglia supply the organs of the abdominopelvic cavity.
The greater splanchnic nerve, formed by the...
01:37Hypothalamic-Pituitary Axis
01:22Target Cell Response to Hormones
Notably, the cellular response can be regulated by altering the number of receptors expressed in the cell. For example, prolonged exposure to elevated hormone levels results in a gradual decline or down-regulation in the number of receptors for that specific hormone on the cell surface. Conversely, in response to low hormone levels, cells may use up-regulation, producing an...