Jove
Visualize
Contact Us

Related Concept Videos

JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies
  1. Home
  3. Benchmarking Of Large Language Models For The Dental Admission Test.
  1. Home
  3. Benchmarking Of Large Language Models For The Dental Admission Test.

Related Experiment Video

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

466

Benchmarking of Large Language Models for the Dental Admission Test.

Yu Hou1,2, Jay Patel3, Liya Dai4

  • 1Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, USA.

Health Data Science
|April 2, 2025

View abstract on PubMed

Summary
This summary is machine-generated.

Large language models (LLMs) show potential for dental school admissions test preparation, excelling in knowledge recall but struggling with visual-spatial reasoning and complex problem-solving. Further development is needed for comprehensive support.

More Related Videos

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.3K
Author Spotlight: 3D Movement Assessment of Maxillary Posterior Teeth in Clear Aligner Treatment
07:32

Author Spotlight: 3D Movement Assessment of Maxillary Posterior Teeth in Clear Aligner Treatment

Published on: February 23, 2024

977

Related Experiment Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

466
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.3K
Author Spotlight: 3D Movement Assessment of Maxillary Posterior Teeth in Clear Aligner Treatment
07:32

Author Spotlight: 3D Movement Assessment of Maxillary Posterior Teeth in Clear Aligner Treatment

Published on: February 23, 2024

977

Area of Science:

  • Artificial Intelligence in Education
  • Medical Education Technology
  • Cognitive Science

Background:

  • Large language models (LLMs) show promise in educational settings.
  • Their performance on high-stakes admissions tests like the Dental Admission Test (DAT) is not well-understood.
  • Assessing LLM capabilities is crucial for evaluating their role in test preparation.

Purpose of the Study:

  • To evaluate the performance of 16 diverse large language models on a sample Dental Admission Test (DAT).
  • To identify strengths and weaknesses of various LLMs in different DAT sections.
  • To understand the challenges LLMs face in processing complex cognitive tasks relevant to dental admissions.

Main Methods:

  • Evaluated 16 LLMs, including general-purpose, domain-specific, and open-source models.
  • Conducted quantitative analysis of model accuracy across DAT sections.
  • Performed qualitative thematic analysis by subject matter experts to identify specific challenges.

    • Main Results:

    • GPT-4o and GPT-o1 excelled in text-based knowledge and comprehension sections (Natural Sciences, Reading Comprehension).
    • Open-source models like Llama3-70B showed competitive performance in Reading Comprehension.
    • All models struggled significantly with Perceptual Ability (image-based tasks), and fine-tuned medical models underperformed in critical thinking.

    Conclusions:

    • LLMs can aid in reinforcing factual knowledge for prospective dental students.
    • Limitations in higher-order cognitive tasks and image-based reasoning necessitate guided integration with human instruction.
    • Future innovations are needed to enhance LLM performance across all DAT cognitive skills.