JoVE - Journal of Visualized Experiments
JoVE Visualize
Contact Us

Comparative analysis of ChatGPT-4o mini, ChatGPT-4o and Gemini Advanced in the treatment of postmenopausal osteoporosis

  • 0Clinical College of Neurology, Neurosurgery and Neurorehabilitation, Tianjin Medical University, Tianjin, China.
BMC musculoskeletal disorders +

|

April 16, 2025

Abstract

BACKGROUND

Osteoporosis is a sex-specific disease. Postmenopausal osteoporosis (PMOP) has been the focus of public health research worldwide. The purpose of this study is to evaluate the quality and readability of artificial intelligence large-scale language models (AI-LLMs): ChatGPT-4o mini, ChatGPT-4o and Gemini Advanced for responses generated in response to questions related to PMOP.

METHODS

We collected 48 PMOP frequently asked questions (FAQs) through offline counseling and online medical community forums. We also prepared 24 specific questions about PMOP based on the Management of Postmenopausal Osteoporosis: 2022 ACOG Clinical Practice Guideline No. 2 (2022 ACOG-PMOP Guideline). In this project, the FAQs were imported into the AI-LLMs (ChatGPT-4o mini, ChatGPT-4o, Gemini Advanced) and randomly assigned to four professional orthopedic surgeons, who independently rated the satisfaction of each response via a 5-point Likert scale. Furthermore, a Flesch Reading Ease (FRE) score was calculated for each of the LLMs' responses to assess the readability of the text generated by each LLM.

RESULTS

When it comes to addressing questions related to PMOP and the 2022 ACOG-PMOP guidelines, ChatGPT-4o and Gemini Advanced provide more concise answers than ChatGPT-4o mini. In terms of the overall FAQs of PMOP, ChatGPT-4o has a significantly higher accuracy rate than ChatGPT-4o mini and Gemini Advanced. When answering questions related to the 2022 ACOG-PMOP guidelines, ChatGPT-4o mini vs. ChatGPT-4o have significantly higher response accuracy than Gemini Advanced. ChatGPT-4o mini, ChatGPT-4o, and Gemini Advanced all have good levels of self-correction.

CONCLUSIONS

Our research shows that Gemini Advanced and ChatGPT-4o provide more concise and intuitive answers. ChatGPT-4o responds better in answering frequently asked questions related to PMOP. When answering questions related to the 2022 ACOG-PMOP guidelines, ChatGPT-4o mini and ChatGPT-4o responded significantly better than Gemini Advanced. ChatGPT-4o mini, ChatGPT-4o, and Gemini Advanced have demonstrated a strong ability to self-correct.

CLINICAL TRIAL NUMBER

Not applicable.

Keywords:

Related Concept Videos

Osteoclasts in Bone Remodeling 01:31

2.8K

Osteoclasts are cells responsible for bone resorption and remodeling. They originate from hematopoietic progenitor cells present in the bone marrow. Numerous progenitor cells fuse to form multinucleated cells, each with 10-20 nuclei. A single osteoclast has a diameter of 150 to 200 µM. These cells have ruffled borders that break down the underlying bone tissue and release minerals such as calcium into the blood in bone resorption. Osteoclasts cling to bones with their ruffled edges during...

Bone Disorders 01:29

3.3K

Aging and its effect on bone remodeling is the most common cause of bone disorders. In young and healthy people, bone deposition and resorption happen at an equal rate to maintain optimal bone health.
Bone deposition is also affected by the levels of sex hormones like estrogen and testosterone that promote osteoblast activity and bone matrix synthesis. When the level of these hormones decreases due to aging, it causes a reduction in bone deposition. As a result, bone resorption by osteoclasts...

Bone Remodeling 01:40

38.0K

Bone remodeling is a continuous and balanced process of bone resorption by osteoclasts and bone formation by osteoblasts. In adults, it helps maintain bone mass and calcium homeostasis. While mechanical stress can stimulate turnover as part of the normal maintenance and reparative process, several hormones also regulate bone remodeling.

Hormonal Control of Bone Remodeling

Parathyroid hormone (PTH) maintains homeostatic control of blood calcium levels by regulating bone resorption. PTH is...

Menopause 01:28

96

Menopause, a natural biological process marking the end of a woman's fertility, typically occurs between the fifth and sixth decade of life. This phase is characterized by the exhaustion of the ovarian follicle pool, leading to less responsive ovaries despite the high levels of Follicle Stimulating Hormone (FSH) and Luteinizing Hormone (LH). The consequential decrease in estrogen production results in symptoms like hot flashes, heavy sweating, headaches, hair loss, muscle pains, vaginal...