Related Experiment Video
Updated: Feb 26, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Performance of Enhanced Large Language Models on Prosthodontic Multiple-Choice Questions
Shenghan Gao1, Zi-Ang Wang2, Zihan Gao1
1Department of Prosthodontics, Peking University School and Hospital of Stomatology & National Center for Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & NHC Key Laboratory of Digital Stomatology & NMPA Key Laboratory for Dental Materials, Beijing, PR China.
Objectives:
To evaluate the performance of retrieval-augmented generation (RAG), in-context learning (ICL) and majority voting-enhanced large language models (LLMs) on prosthodontic questions.
Methods:
Two LLMs, OpenAI o1 and DeepSeek-R1, were used as base models in this study. The base models were enhanced using RAG, ICL and majority voting. The resulting enhanced LLMs-enhanced DeepSeek-R1 and enhanced OpenAI o1-were evaluated alongside their original counterparts. Standardised Chinese-language and English-language prosthodontics questions were used to evaluate the performance of the base and enhanced LLMs. The correctness of each answer and the error types were recorded. χ2 tests were used to compare differences among the 4 LLMs (α = 0.05).
Results:
Enhanced versions of both DeepSeek-R1 and OpenAI o1 achieved significant accuracy improvement compared with their base versions in Chinese-language multiple-choice questions (C-MCQs) (P < .001). In English-language multiple-choice questions (E-MCQs), both enhanced versions also exhibited higher accuracy, but the difference was not statistically significant (P = .145). In error analysis, enhanced LLMs significantly reduced knowledge-based errors in C-MCQs (P < .008) but showed only a non-significant reduction in E-MCQs (P = .604). The results reveal that the significant improvement gains from model enhancement were confined to C-MCQs and were not observed in E-MCQs.
Conclusions:
The LLMs enhanced using inference-time strategies including RAG, ICL and majority voting show significantly improved accuracy in answering C-MCQs. Although this approach effectively mitigates errors, our analysis reveals a lack of consistent statistically significant improvements in E-MCQs.
Clinical Significance:
This study illustrates the performance of enhanced LLMs in analysing prosthodontics questions with good accuracy and potential in dentistry education. This implies that enhanced LLMs may be more suitable than non-enhanced ones for processing dental-related tasks.
More Related Videos
07:32Author Spotlight: 3D Movement Assessment of Maxillary Posterior Teeth in Clear Aligner Treatment
Published on: February 23, 2024
09:00Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
Published on: August 16, 2024
