大型语言模型的基准测试用于牙科招生考试
Yu Hou1,2, Jay Patel3, Liya Dai4
1Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, USA.
Health data science
|April 2, 2025
概括
大型语言模型 (LLM) 显示出牙科学校招生考试准备的潜力,在知识回忆方面表现出色,但在视觉空间推理和复杂问题解决方面扎. 为了提供全面的支持,需要进一步发展.
科学领域:
- 教育中的人工智能
- 医疗教育 技术 技术 医学教育
- 认知科学 认知科学
背景情况:
- 大型语言模型 (LLM) 在教育环境中表现有前途.
- 他们在像牙科招生考试 (DAT) 这样的高风险招生考试中的表现并未得到充分理解.
- 评估LLM能力对于评估其在考试准备中的作用至关重要.
研究的目的:
- 评估16个不同的大型语言模型在样本牙科招生考试 (DAT) 上的表现.
- 在不同的DAT段中识别各种LLM的优缺点.
- 了解LLM在处理牙科招生相关的复杂认知任务时所面临的挑战.
主要方法:
- 评估了16个LLM,包括通用,特定领域和开源模型.
- 对DAT段的模型准确性进行了定量分析.
- 由主题专家进行定性主题分析,以确定具体的挑战.
主要成果:
- 在基于文本的知识和理解部分 (自然科学,阅读理解) 中,GPT-4o和GPT-o1表现出色.
- 像Llama3-70B这样的开源模型在阅读理解方面表现出了竞争力.
- 所有模型都在感知能力 (基于图像的任务) 上有显著的困难,精心调整的医疗模型在批判性思维方面表现不佳.
结论:
- 士学位可以帮助未来的牙科学生加强事实知识.
- 高级认知任务和基于图像的推理的局限性需要与人类指令进行指导整合.
- 需要未来的创新来提高LLM在所有DAT认知技能中的表现.
相关概念视频
01:15Teeth
The formation of teeth, also known as odontogenesis, is a complex process that begins in utero, around the sixth week of embryonic development. There are three stages to this process: the bud stage, the cap stage, and the bell stage.
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin and...
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin and...
01:21Tooth Anatomy
The human tooth enables us to eat a variety of foods, speak clearly, and even aid in shaping our faces. Teeth are composed of various elements that work together. Here's a detailed look at the anatomy of a human tooth.
The Crown, Neck, and Root
The visible part of the tooth is referred to as the crown. It's covered by enamel, the hardest substance in the human body. The crown is uniquely shaped for each type of tooth, allowing for different functions such as cutting, tearing, or grinding food.
The Crown, Neck, and Root
The visible part of the tooth is referred to as the crown. It's covered by enamel, the hardest substance in the human body. The crown is uniquely shaped for each type of tooth, allowing for different functions such as cutting, tearing, or grinding food.
01:26Assessment of the Mouth
A thorough mouth assessment, including inspection and palpation of the lips, gums, tongue, tonsils, uvula, and pharynx, is crucial in detecting potential health issues. Diseases ranging from oral cancer to systemic conditions like diabetes could be identified early through careful oral examination. This article provides a detailed guide on conducting a comprehensive mouth assessment.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.