Related Experiment Video
Updated: Jun 10, 2025
09:10
Digital Hybrid Model Preparation for Virtual Planning of Reconstructive Dentoalveolar Surgical Procedures
Published on: August 5, 2021
1.7K
ScholarGPT's performance in oral and maxillofacial surgery
1Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Sivas Cumhuriyet University, Sivas 58000, Turkiye.
Journal of Stomatology, Oral and Maxillofacial Surgery
|October 10, 2024
Summary
Scholar GPT significantly outperformed ChatGPT in answering oral and maxillofacial surgery questions, providing more accurate and consistent AI-generated information. This suggests specialized GPT models enhance reliability in technical fields.
Area of Science:
- Oral and Maxillofacial Surgery
- Artificial Intelligence in Medicine
- Natural Language Processing
Background:
- The increasing use of AI necessitates evaluating its performance in specialized medical fields.
- Previous studies assessed general AI models like ChatGPT for medical queries.
Purpose of the Study:
- To evaluate Scholar GPT's performance on technical oral and maxillofacial surgery questions.
- To compare Scholar GPT's efficacy against ChatGPT.
Main Methods:
- 60 technical questions from oral and maxillofacial surgery were used.
- Scholar GPT responses were evaluated using a modified Global Quality Scale (GQS).
- Scoring involved randomization and multiple evaluations by a single researcher.
Main Results:
- Scholar GPT achieved a high average GQS score of 4.48.
- This score was statistically significantly higher than ChatGPT's previous average of 3.1 (p < 0.001).
- No significant differences were found across different surgical topic groups.
Conclusions:
- Scholar GPT demonstrates superior performance and consistency over ChatGPT for oral and maxillofacial surgery queries.
- AI models trained on academic data offer more reliable medical information.
- Developing specialized GPTs for surgery can improve AI-generated content quality.
