Can Large Language Models Serve as Reliable Tools for Information in Dentistry? A Systematic Review

Area of Science:

• Dental Education
• Artificial Intelligence in Healthcare
• Information Science

Background:

• Large language models (LLMs) are increasingly utilized by dental students for subject-specific information retrieval.
• Widespread adoption of LLMs in dentistry raises significant concerns regarding the potential for misinformation.
• A critical evaluation of LLM performance in dental education is necessary.

Purpose of the Study:

• To systematically review and critically assess studies evaluating the performance of large language models (LLMs) in the field of dentistry.
• To identify the benefits and limitations of LLMs as educational tools for dental students.
• To provide evidence-based recommendations for the ethical and effective integration of LLMs in dental curricula.

Main Methods:

• A comprehensive systematic search was performed across multiple major scientific databases (PubMed/Medline, Scopus, Embase, Web of Science, Google Scholar, Saudi Digital Library) up to September 2024.
• Included studies were assessed for quality using the Prediction Model Risk of Bias Assessment Tool (PROBAST).
• Data extraction and synthesis were conducted on 31 studies that met the inclusion criteria.

Main Results:

• Out of 2030 identified studies, 31 met the inclusion criteria after screening and deduplication.
• Approximately half of the included studies were classified as "high risk" for bias, while the remainder were "low risk."
• Key limitations identified include LLMs' inability to cite sources and their propensity for generating fabricated citations, though applicability was rated as "low concern."

Conclusions:

• Large language models (LLMs) demonstrate potential as supplementary educational tools in dentistry, possibly enhancing academic performance.
• Concerns regarding inaccuracies and unreliable citations necessitate further research and integration with validated resources.
• LLMs should be adopted as complementary tools in dental education, with a strong emphasis on ethical guidelines and awareness of their limitations.