基于 ортодонтика执照考试的大型语言模型中的知识能力和错误模式的限制
Zhang Ruoyan1,2, Liu Liu1,2, Zhao Qian1,2
1Medical School of Chinese PLA, Beijing, 100853, China.
BMC oral health
|November 22, 2025
概括
在正执照考试中,Deepseek-R1 (DS) 的表现明显优于ChatGPT-4 (GPT),获得了更高的准确性. 然而,这两种大型语言模型 (LLM) 在临床推理方面都有局限性,需要临床医生的验证.
科学领域:
- 人工智能在牙科中的应用
- 医疗教育 技术 技术 医学教育
- ортодонтический评估 评估 ортодонтический评估
背景情况:
- 大型语言模型 (LLM) 越来越多地被用于医学应用.
- 评估LLM在牙科等专业领域的表现对于了解其实用性和局限性至关重要.
- ортодонтика执照考试为评估医学知识和临床推理提供了标准化的基准.
研究的目的:
- 评估两个领先的LLM,Deepseek-R1 (DS) 和ChatGPT-4 (GPT) 的性能限制,在牙科领域.
- 为了比较不同正牙知识领域的LLM绩效,并识别常见的错误类型.
主要方法:
- 总共使用了来自中国国家正统牙科专家执照考试的396个基于文本的问题.
- 问题按知识领域 (基础生物力学,跨学科整合,专业理论,临床决策) 和错误类型 (事实,逻辑,语义) 分类.
- 性能指标包括准确性和错误率在DS和GPT之间进行了比较.
主要成果:
- 与GPT (52.3%) 相比,DS的整体准确度 (80.3%) 显著提高 (p < 0.001).
- 在基础知识 (79.8%对43.4%) 和跨学科领域 (81.0%对53.0%) 中,DS也表现出优异的表现.
- 在这两种模型中,事实错误是最常见的,而DS显示出更高的逻辑错误率.
结论:
- 通过标准化考试,DS在正经知识评估方面表现出比GPT更高的熟练程度.
- 这两种LLM在需要复杂的临床推理的专业牙科领域都存在局限性.
- 虽然DS对人工智能辅助的正牙培训和评估有希望,但由于持续存在的错误和特定领域的弱点,临床医生监督仍然至关重要.
相关概念视频
Language and Cognition
696
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
696
Teeth
1.6K
The formation of teeth, also known as odontogenesis, is a complex process that begins in utero, around the sixth week of embryonic development. There are three stages to this process: the bud stage, the cap stage, and the bell stage.
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin...
In the bud stage, the tooth germ (an aggregation of cells) starts to form in the developing jawbone. During the cap stage, the tooth germ differentiates into enamel organ, dental papilla, and dental sac, which will later develop into the tooth's enamel, dentin...
1.6K
Mechanistic Models: Compartment Models in Individual and Population Analysis
230
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
230
Documentation of Nursing Diagnosis
1.6K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.6K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
267
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
267
Barriers to Effective Communication II
4.5K
The barriers to effective communication also include cultural barriers, semantic barriers, gender barriers, and time constraints.
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...
4.5K


