Related Experiment Video
Updated: Sep 13, 2025
07:42
Quasistatic Mechanical Testing for Computer-Aided Design and Manufacturing Occlusal Veneers Cemented to Milled Dentin Analog Material
Published on: December 20, 2024
462
Evaluation of Chatbot Responses to Text-Based Multiple-Choice Questions in Prosthodontic and Restorative Dentistry
Reinhard Chun Wang Chau1, Khaing Myat Thu1, Ollie Yiru Yu1
1Faculty of Dentistry, The University of Hong Kong, Hong Kong 999077, China.
Dentistry Journal
|July 25, 2025
Summary
GPT-4.0 showed superior accuracy in answering prosthodontic and restorative dentistry questions compared to Claude-2 and Llama-2. This study highlights varying artificial intelligence chatbot efficacy in specialized dental education.
Area of Science:
- Artificial Intelligence in Dentistry
- Dental Education Technology
- Natural Language Processing in Healthcare
Background:
- The integration of artificial intelligence (AI) chatbots into specialized fields like dentistry is rapidly increasing.
- Evaluating the performance of AI chatbots in answering complex dental questions is crucial for understanding their utility.
- This study focuses on prosthodontic and restorative dentistry, key areas within the dental curriculum.
Purpose of the Study:
- To assess the response accuracy and rationale quality of three leading AI chatbots: GPT-4.0, Claude-2, and Llama-2.
- To compare their performance in answering multiple-choice questions from dental board examinations.
- To determine the suitability of these AI tools for dental education and practice.
Main Methods:
- 191 multiple-choice questions from the United States Integrated National Board Dental Examination (INBDE) and the United Kingdom Overseas Registration Examination (ORE) were used.
- AI chatbots (GPT-4.0, Claude-2, Llama-2) answered the questions, and their responses were compared against official answer keys.
- Two dental specialists evaluated the accuracy, relevance, and comprehensiveness of the AI-generated rationales, with high inter-rater reliability (κ = 0.83).
Main Results:
- GPT-4.0 achieved significantly higher accuracy (65.4%) compared to Claude-2 (41.9%) and Llama-2 (26.2%) (p < 0.017).
- GPT-4.0 also demonstrated superior rationale quality ('Correct Answer, Correct Rationale' at 58.1%) versus Claude-2 (37.2%) and Llama-2 (24.1%) (p < 0.017).
- Statistically significant differences in both answer accuracy and rationale quality were observed among all tested AI chatbots (p < 0.001).
Conclusions:
- GPT-4.0 exhibits the highest performance in terms of accuracy and reasoning quality for prosthodontic and restorative dentistry questions.
- The findings indicate a notable variation in the effectiveness of AI chatbots within specialized dental domains.
- Further research is warranted to explore AI's role and limitations in advanced dental education and examination preparation.
Related Concept Videos
01:26Assessment of the Mouth
412
A thorough mouth assessment, including inspection and palpation of the lips, gums, tongue, tonsils, uvula, and pharynx, is crucial in detecting potential health issues. Diseases ranging from oral cancer to systemic conditions like diabetes could be identified early through careful oral examination. This article provides a detailed guide on conducting a comprehensive mouth assessment.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.
412
01:15Teeth
703
The formation of teeth, also known as odontogenesis, is a complex process that begins in utero, around the sixth week of embryonic development. There are three stages to this process: the bud stage, the cap stage, and the bell stage.
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin...
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin...
703